-- Logs begin at Wed 2022-06-15 13:46:35 UTC, end at Wed 2022-06-15 15:18:55 UTC. -- Jun 15 14:29:56.234174 dv-cinder-787296-10-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 15 14:29:56.241747 dv-cinder-787296-10-iscsi devstack@keystone.service[80586]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 15 14:29:56.243770 dv-cinder-787296-10-iscsi devstack@keystone.service[80586]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Jun 15 14:29:56 2022] *** Jun 15 14:29:56.243770 dv-cinder-787296-10-iscsi devstack@keystone.service[80586]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 15 14:29:56.243770 dv-cinder-787296-10-iscsi devstack@keystone.service[80586]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 15 14:29:56.243770 dv-cinder-787296-10-iscsi devstack@keystone.service[80586]: nodename: dv-cinder-787296-10-iscsi Jun 15 14:29:56.243770 dv-cinder-787296-10-iscsi devstack@keystone.service[80586]: machine: x86_64 Jun 15 14:29:56.243770 dv-cinder-787296-10-iscsi devstack@keystone.service[80586]: clock source: unix Jun 15 14:29:56.243770 dv-cinder-787296-10-iscsi devstack@keystone.service[80586]: pcre jit disabled Jun 15 14:29:56.244277 dv-cinder-787296-10-iscsi devstack@keystone.service[80586]: detected number of CPU cores: 4 Jun 15 14:29:56.244277 dv-cinder-787296-10-iscsi devstack@keystone.service[80586]: current working directory: / Jun 15 14:29:56.244277 dv-cinder-787296-10-iscsi devstack@keystone.service[80586]: detected binary path: /usr/bin/uwsgi-core Jun 15 14:29:56.244277 dv-cinder-787296-10-iscsi devstack@keystone.service[80586]: your processes number limit is 47680 Jun 15 14:29:56.244277 dv-cinder-787296-10-iscsi devstack@keystone.service[80586]: your memory page size is 4096 bytes Jun 15 14:29:56.244277 dv-cinder-787296-10-iscsi devstack@keystone.service[80586]: detected max file descriptor number: 2048 Jun 15 14:29:56.244277 dv-cinder-787296-10-iscsi devstack@keystone.service[80586]: lock engine: pthread robust mutexes Jun 15 14:29:56.244277 dv-cinder-787296-10-iscsi devstack@keystone.service[80586]: thunder lock: enabled Jun 15 14:29:56.245237 dv-cinder-787296-10-iscsi devstack@keystone.service[80586]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 15 14:29:56.245378 dv-cinder-787296-10-iscsi devstack@keystone.service[80586]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 15 14:29:56.292709 dv-cinder-787296-10-iscsi devstack@keystone.service[80586]: Python main interpreter initialized at 0x558b400169c0 Jun 15 14:29:56.292709 dv-cinder-787296-10-iscsi devstack@keystone.service[80586]: python threads support enabled Jun 15 14:29:56.292709 dv-cinder-787296-10-iscsi devstack@keystone.service[80586]: your server socket listen backlog is limited to 100 connections Jun 15 14:29:56.292709 dv-cinder-787296-10-iscsi devstack@keystone.service[80586]: your mercy for graceful operations on workers is 90 seconds Jun 15 14:29:56.293088 dv-cinder-787296-10-iscsi devstack@keystone.service[80586]: mapped 403077 bytes (393 KB) for 2 cores Jun 15 14:29:56.293163 dv-cinder-787296-10-iscsi devstack@keystone.service[80586]: *** Operational MODE: preforking *** Jun 15 14:29:56.293432 dv-cinder-787296-10-iscsi devstack@keystone.service[80586]: *** uWSGI is running in multiple interpreter mode *** Jun 15 14:29:56.293432 dv-cinder-787296-10-iscsi devstack@keystone.service[80586]: spawned uWSGI master process (pid: 80586) Jun 15 14:29:56.293637 dv-cinder-787296-10-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 15 14:29:56.296779 dv-cinder-787296-10-iscsi devstack@keystone.service[80586]: spawned uWSGI worker 1 (pid: 80600, cores: 1) Jun 15 14:29:56.296779 dv-cinder-787296-10-iscsi devstack@keystone.service[80586]: spawned uWSGI worker 2 (pid: 80601, cores: 1) Jun 15 14:29:56.296779 dv-cinder-787296-10-iscsi devstack@keystone.service[80586]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 15 14:30:01.840355 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 15 14:30:01.840355 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: from cryptography.utils import int_from_bytes Jun 15 14:30:01.841101 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 15 14:30:01.841101 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: from cryptography.utils import int_from_bytes Jun 15 14:30:01.849297 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 15 14:30:01.849297 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: from cryptography.utils import int_from_bytes Jun 15 14:30:01.849760 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 15 14:30:01.849760 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: from cryptography.utils import int_from_bytes Jun 15 14:30:01.891010 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 15 14:30:01.891224 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 15 14:30:01.891439 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] command line args: [] {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 15 14:30:01.891710 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 15 14:30:01.891887 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ================================================================================ {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 15 14:30:01.892158 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] admin_token = **** {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.892406 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] config_dir = [] {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.892592 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.892771 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] config_source = [] {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.892948 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] debug = True {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.893152 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.894354 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.894354 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.894354 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.894354 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.894354 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] list_limit = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.894652 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] log_config_append = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.894710 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.894889 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] log_dir = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.895063 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] log_file = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.895279 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.895525 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.895769 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.896011 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.896210 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.896385 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.896625 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.896866 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.897095 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.897325 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.897555 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.897786 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.898028 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.898269 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.898539 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.898682 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 15 14:30:01.898880 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 15 14:30:01.899021 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.42/identity {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.899107 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] command line args: [] {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 15 14:30:01.899200 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] publish_errors = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.899333 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 15 14:30:01.899419 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.899512 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ================================================================================ {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 15 14:30:01.899641 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.899765 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] admin_token = **** {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.899843 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.900015 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] config_dir = [] {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.900121 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.900237 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.900318 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.900422 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] config_source = [] {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.900501 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] standard_threads = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.900604 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] debug = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.900732 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.900817 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.900973 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.901054 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.901133 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.901211 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.901544 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] use_journal = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.901712 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.901791 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] use_json = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.901867 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] use_stderr = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.901946 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.902024 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] use_syslog = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.902180 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] list_limit = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.902262 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.902361 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] log_config_append = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.902489 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.902577 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.902704 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] log_dir = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.902788 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.902882 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] log_file = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.902965 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.903118 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.903266 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.903369 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.903448 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.903621 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.903704 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.903870 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.904016 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.904129 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.904216 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.904295 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.904375 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.904482 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.904587 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.904731 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.904833 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.904966 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.905056 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.905201 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.905285 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.905434 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.905512 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.905668 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.905756 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.905916 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.906002 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.906162 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.906246 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.906427 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.907102 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.907206 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.907439 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.907672 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.907776 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.42/identity {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.907924 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.908044 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] publish_errors = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.908215 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.908305 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.908467 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.908547 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.908713 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.908799 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.908940 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.909019 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.909167 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.909247 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.909393 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.909473 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] standard_threads = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.909629 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.909707 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.909872 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.909961 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.910033 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.910101 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.910216 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] use_journal = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.910330 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.910418 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] use_json = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.910558 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.910637 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] use_stderr = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.910733 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] use_syslog = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.910810 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.910968 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:30:01.911059 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.911228 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.911312 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.911469 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.911553 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.911700 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.911784 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.911934 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.912014 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.912189 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.912272 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.912419 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.912572 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.912648 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.912719 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.912861 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.912939 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.913087 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.913177 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.913320 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.913409 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.913568 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.913652 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.913796 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.913882 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.914030 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.914115 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.914258 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.914343 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.914491 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.914578 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.914724 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.914802 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.914970 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.915054 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.915197 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.915317 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.915416 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.915519 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.915652 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.915762 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.915884 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.916002 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.916124 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.916297 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.916373 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.916569 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.916666 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.916813 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.916927 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.917046 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.917126 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.917280 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.917409 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.917494 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.917568 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.917729 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.917809 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.917966 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.918055 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.918192 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.918273 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.918424 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.918505 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.918655 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.918746 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.918892 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.918982 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.919138 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.919217 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.919387 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.919465 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.919626 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.919711 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.919859 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.920014 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.920175 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.920272 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.920348 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.920474 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.920552 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.920634 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.920787 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.920867 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.921016 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.921106 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.921249 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.921329 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.921482 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.921570 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.921715 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.921805 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.921944 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.922022 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.922234 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.922311 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.923071 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.923170 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.923335 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.923620 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.923713 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.923787 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.923857 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.924023 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.924153 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.924289 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.924371 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.924525 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.924605 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.924760 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.924841 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.924993 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.925072 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.925220 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.925306 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.925460 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.925535 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.925661 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.925740 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.925891 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.925979 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.926119 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.926214 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.926354 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.926434 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.926586 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.926670 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.926832 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.926920 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.927066 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.927156 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.927318 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.927403 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.927553 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.927795 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.928036 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.928346 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.928590 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.928845 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.929095 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.929337 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.929577 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.929817 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.930069 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.930311 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.930549 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.930794 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.931033 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.931272 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.931512 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.931751 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.931990 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.932288 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.932535 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.932781 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.933023 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.933268 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.933508 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.933747 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.933986 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.934220 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.934460 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.934699 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.934939 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.935173 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.935418 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.935658 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.935913 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.936188 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.936454 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.936722 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.936967 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.937204 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] auth.external = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.937437 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.937740 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.938011 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.938401 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] auth.password = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.938842 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] auth.token = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.939157 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:01.939403 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.013271 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.013562 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.013826 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] auth.external = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.014085 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.014411 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.014661 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.014907 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] auth.password = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.015152 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] auth.token = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.015403 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.015651 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.016485 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.016893 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.017322 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.017727 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.018048 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.018295 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] credential.caching = True {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.018554 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.018816 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.019078 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.019320 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.019570 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.019811 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.020069 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.020350 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.020590 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.020847 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.021141 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.021392 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.021660 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.021907 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.022172 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.022412 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] federation.caching = True {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.022650 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.022899 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.023151 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.023385 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.023635 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.023875 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.024153 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.024396 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.024655 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.024893 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.025141 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.025378 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] identity.caching = True {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.025626 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.025787 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.025868 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.026038 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.026128 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.026573 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.026660 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.026822 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.026904 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.027051 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.027133 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.027288 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.027369 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] credential.caching = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.027540 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.027621 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.027792 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.027873 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.028024 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.028126 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.028279 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.028362 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.028510 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.028597 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.028756 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.028838 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.029010 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.029091 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.029258 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.029362 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.029513 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.029614 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.029759 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.029882 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.030032 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.030181 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.030261 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.030440 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.030528 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.030709 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.030789 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.030945 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.031025 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.031175 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.031262 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.031409 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.031493 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] federation.caching = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.032213 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.032409 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.032491 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.032694 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.032889 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.032970 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.033163 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.033298 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.033408 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.033550 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.033633 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.033816 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.033897 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.034069 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.034149 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.034321 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.034402 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.034556 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.034638 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.034786 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.034872 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.035022 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.035103 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.035258 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.035339 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.035503 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.035592 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] identity.caching = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.035739 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.035819 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.035975 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.036055 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.036245 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.036345 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.036493 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.036575 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.036723 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.036805 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.036953 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.037041 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.037201 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.037282 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.037449 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.037530 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.037684 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.037774 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.037917 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.037998 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.038151 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.038231 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.038380 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.user = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.038472 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.038619 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.038700 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.038871 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.038951 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.039104 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.039192 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.039351 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.039432 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.039595 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.039676 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.039839 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.039928 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.040072 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.040172 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.040325 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.040405 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.040558 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.040650 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.040791 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.040872 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.041024 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.041104 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.041250 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.041339 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.041495 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.041576 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.041725 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.041806 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.041965 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.042054 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.042203 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.042282 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.042444 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.042523 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.042669 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.042758 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.042914 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.042995 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.043149 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.043229 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.043383 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.043476 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.043616 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.043697 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.043848 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.043928 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.044081 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.044221 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.044346 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.044427 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.044580 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.044658 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.044830 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.044919 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.045056 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.045136 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.045300 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.045379 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.045531 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.045619 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.045765 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.045845 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.046000 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.046080 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.046235 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.046322 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.046472 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.046552 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.046701 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.046781 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.046936 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.047024 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.047204 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] resource.caching = True {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.047339 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.047445 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.047561 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.047684 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.047916 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.047998 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.user = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.048225 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.048376 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.048505 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.048641 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.048768 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.048893 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.049019 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.049144 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.049269 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.049403 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.049529 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.049654 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.049778 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] role.caching = True {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.049902 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.050028 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] role.driver = sql {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.050235 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.050368 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.050494 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.050643 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.050769 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.050913 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.051064 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.051145 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.051219 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.051290 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.051362 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.051434 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.051514 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.051588 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.051661 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.051733 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.051804 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.051876 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.052060 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.052186 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.052319 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.052400 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.052570 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.052650 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.052809 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.052891 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.053051 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.053131 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.053293 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.053483 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.053572 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.053720 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.053800 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.053969 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.054050 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.054218 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.054307 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.054464 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.054545 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.054707 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.054787 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.054938 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.055027 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.055187 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.055266 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.055428 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.055508 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.055663 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.055751 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.055902 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.055989 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.056156 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.056242 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.056389 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.056480 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.056627 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.056708 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.056872 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] resource.caching = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.056952 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.057117 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.057198 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.057386 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.057519 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.057649 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.057779 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.057914 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.058040 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.058176 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] token.caching = True {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.058301 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.058426 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.058550 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.058683 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.058809 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.058934 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.059058 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.059182 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.059306 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.059438 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.059563 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] role.caching = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.059688 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.059813 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] role.driver = sql {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.059947 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.060073 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.060257 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.060384 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.060508 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.060633 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.060759 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.060883 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.061016 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.061141 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.061266 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.061391 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.061521 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.061646 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.061788 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.061914 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.062039 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.062164 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.062288 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.062412 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.062552 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.062677 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.062828 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] database.connection = **** {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.063020 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.063164 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.063290 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.063415 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.063559 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.063685 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.063685 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.063685 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.064037 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.064191 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.064328 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.064453 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.064578 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.064703 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.064828 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.064953 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.065086 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.065211 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.065345 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.065471 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.065595 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.065719 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.065851 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.065977 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.065977 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.066212 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.066337 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.066462 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.066594 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.066719 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.066843 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.066977 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.067102 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.067227 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.067360 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 15 14:30:02.067491 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.067491 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.067491 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.067491 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.067950 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.067950 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.068215 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] token.caching = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.068344 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.068572 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.068816 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.069080 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.069333 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.069575 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.069825 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.070073 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.070328 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.070569 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.070811 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.071045 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.071296 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.071549 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.071782 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.072030 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.072353 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.072621 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] database.connection = **** {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.072872 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.073127 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.073369 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.073616 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.073857 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.074098 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.074338 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.074584 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.074833 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.075071 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.075319 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.075560 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.075813 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.076054 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.076330 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.076569 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.076816 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.077057 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:30:02.077230 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 15 14:30:02.096853 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:30:02.097030 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:30:02.097239 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:30:02.103519 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:30:02.103756 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:30:02.103999 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:30:02.105515 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:30:02.112859 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:30:02.133209 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:30:02.133438 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:30:02.139036 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:30:02.139267 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:30:02.186960 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:30:02.188989 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:30:02.235654 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:30:02.235878 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:30:02.238733 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:30:02.238971 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:30:02.301724 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:30:02.301948 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:30:02.302168 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:30:02.302289 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:30:02.302375 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:30:02.302526 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:30:02.302619 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:30:02.302754 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:30:02.302838 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:30:02.302978 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:30:02.303198 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:30:02.303413 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:30:02.325459 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:30:02.325685 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:30:02.325856 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:30:02.325949 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:30:02.326088 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:30:02.326197 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:30:02.326319 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:30:02.326403 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:30:02.326533 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:30:02.326617 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:30:02.326746 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:30:02.326831 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:30:02.326962 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:30:02.327178 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:30:02.338165 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:30:02.338836 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:30:02.351440 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:30:02.351628 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:30:02.351847 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:30:02.351962 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:30:02.352052 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:30:02.352224 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:30:02.352314 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:30:02.352451 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:30:02.352536 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:30:02.352677 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:30:02.352771 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80601) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:30:02.352901 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:30:02.353121 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:30:02.353337 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:30:02.369608 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.369816 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 15 14:30:02.370210 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.370301 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.370420 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 15 14:30:02.370504 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 15 14:30:02.370750 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.370840 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.370950 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 15 14:30:02.371049 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.371251 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.371398 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.371502 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.371665 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.371930 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.372220 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.382880 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.383151 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.383414 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.383687 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.383902 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.384204 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.384477 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.384760 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.392540 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:30:02.393787 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80600) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:30:02.395793 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:30:02.396070 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.396439 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.396714 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.396990 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.397207 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80600) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:30:02.397291 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.397492 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.397583 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.397756 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.397838 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.398016 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.398097 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.398279 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.398361 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.398538 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.398620 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.398807 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.399079 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.399343 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.399613 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.399877 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.427050 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.427366 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.428671 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:30:02.428949 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.429099 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80600) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:30:02.429379 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.434141 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:30:02.434417 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.434649 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80600) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:30:02.434733 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.434930 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.435211 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.442662 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:30:02.442941 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.443489 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80600) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:30:02.443773 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.447383 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:30:02.447666 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.447941 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.448242 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.448325 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80600) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:30:02.448513 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.448594 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.448780 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.448863 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.449043 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.449127 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.449300 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.449382 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.449595 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.449868 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.450138 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.468421 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.468692 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.470028 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.470304 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.473260 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:30:02.474828 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80600) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:30:02.476889 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.477180 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.478472 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.478768 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.482252 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:30:02.483813 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80600) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:30:02.486175 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:30:02.487716 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80600) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:30:02.489788 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.490082 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.490361 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.490636 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.490907 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.491188 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.491440 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.491739 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.492024 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.492343 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.492620 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.492898 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.512045 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:30:02.512355 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.512618 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.512877 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.514001 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80600) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:30:02.514287 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.514556 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.514819 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.521362 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.523049 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.523327 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.523573 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.525330 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.525609 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.526863 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:30:02.527148 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.527424 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.527710 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.527994 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.528972 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80600) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:30:02.529252 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.529663 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.529964 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.530249 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.541261 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:30:02.541540 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.541814 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.542088 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.542363 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.542628 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.542891 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.543958 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80600) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:30:02.544271 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.544558 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.544838 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.545116 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.545401 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.545675 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.560786 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:30:02.564540 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:30:02.564839 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80600) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:30:02.567785 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.568114 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80600) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:30:02.569395 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.571151 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:30:02.571421 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.571507 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.571694 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.573059 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.574831 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80600) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:30:02.575113 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.575393 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.579460 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.581092 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:30:02.583664 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.584318 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.584603 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.584880 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.585147 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.585313 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80600) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:30:02.588638 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.588931 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.589220 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.589504 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.594562 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:30:02.594843 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.595120 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.599244 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80600) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:30:02.599535 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.599826 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.602962 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80601) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:30:02.603194 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.603466 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.603734 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.604006 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.604304 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.604581 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.604850 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.605214 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.605478 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.605743 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.606006 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.606266 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.606521 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80601) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.607675 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80600) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:30:02.607951 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.608256 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.608529 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.608807 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.609102 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.609373 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.609762 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.610135 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.610593 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.610870 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.611152 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.611424 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.611689 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:30:02.644977 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x558b400169c0 pid: 80601 (default app) Jun 15 14:30:02.648431 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df3eba-7198-420a-a052-f9188aecff8a None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:02.648731 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df3eba-7198-420a-a052-f9188aecff8a None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:02.649008 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df3eba-7198-420a-a052-f9188aecff8a None None] PATH_INFO: `/v3` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:02.650837 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x558b400169c0 pid: 80600 (default app) Jun 15 14:30:02.653471 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1/1] 10.222.0.42 () {50 vars in 791 bytes} [Wed Jun 15 14:30:02 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:11.086202 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db77505-3596-4241-8786-e13ae60fe3ef None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:11.086813 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db77505-3596-4241-8786-e13ae60fe3ef None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:11.086813 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db77505-3596-4241-8786-e13ae60fe3ef None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:11.091142 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1/2] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:30:11 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:11.096460 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ffbb5-9821-4abb-a5fb-9b5b7245b416 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:11.096903 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ffbb5-9821-4abb-a5fb-9b5b7245b416 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:11.097022 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ffbb5-9821-4abb-a5fb-9b5b7245b416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:11.166466 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: INFO dbcounter [None req-594ffbb5-9821-4abb-a5fb-9b5b7245b416 None None] Registered counter for database keystone Jun 15 14:30:11.188926 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG oslo_db.sqlalchemy.engines [None req-594ffbb5-9821-4abb-a5fb-9b5b7245b416 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80601) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 15 14:30:11.195198 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writer thread running {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 15 14:30:11.373167 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG passlib.handlers.bcrypt [None req-594ffbb5-9821-4abb-a5fb-9b5b7245b416 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80601) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 15 14:30:11.373514 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG passlib.handlers.bcrypt [None req-594ffbb5-9821-4abb-a5fb-9b5b7245b416 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80601) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 15 14:30:11.429693 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-594ffbb5-9821-4abb-a5fb-9b5b7245b416 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:11.431317 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-594ffbb5-9821-4abb-a5fb-9b5b7245b416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:11.555453 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 2/3] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:11 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 460 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 15 14:30:11.711536 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db77505-3596-4241-8786-e13ae60fe3ef None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:11.782093 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: INFO dbcounter [None req-5db77505-3596-4241-8786-e13ae60fe3ef None None] Registered counter for database keystone Jun 15 14:30:11.800997 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG oslo_db.sqlalchemy.engines [None req-5db77505-3596-4241-8786-e13ae60fe3ef None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80600) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 15 14:30:11.807573 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writer thread running {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 15 14:30:11.928318 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db77505-3596-4241-8786-e13ae60fe3ef None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:11.932549 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8366b80d-7a6f-4133-9aa8-d0fc10811f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:11.933208 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8366b80d-7a6f-4133-9aa8-d0fc10811f91 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:11.933501 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8366b80d-7a6f-4133-9aa8-d0fc10811f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:11.933774 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8366b80d-7a6f-4133-9aa8-d0fc10811f91 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:11.942462 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8366b80d-7a6f-4133-9aa8-d0fc10811f91 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:11.994480 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: WARNING keystone.common.rbac_enforcer.enforcer [None req-8366b80d-7a6f-4133-9aa8-d0fc10811f91 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 15 14:30:11.994770 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8366b80d-7a6f-4133-9aa8-d0fc10811f91 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:12.005370 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 2/4] 10.222.0.42 () {58 vars in 1324 bytes} [Wed Jun 15 14:30:11 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 15 14:30:12.011457 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594ffbb5-9821-4abb-a5fb-9b5b7245b416 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:12.027926 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594ffbb5-9821-4abb-a5fb-9b5b7245b416 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:12.032544 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0335b0-0079-42db-9f6d-fd53e986a411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:12.033649 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0335b0-0079-42db-9f6d-fd53e986a411 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:12.033933 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0335b0-0079-42db-9f6d-fd53e986a411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:12.034207 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0335b0-0079-42db-9f6d-fd53e986a411 admin admin] PATH_INFO: `/v3/projects/688b80ae31764832aed9d6e0f3d3660f` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:12.035690 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0335b0-0079-42db-9f6d-fd53e986a411 admin admin] RBAC: Authorizing `identity:get_project(project_id=688b80ae31764832aed9d6e0f3d3660f)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:12.086894 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: WARNING keystone.common.rbac_enforcer.enforcer [None req-9d0335b0-0079-42db-9f6d-fd53e986a411 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 15 14:30:12.087238 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0335b0-0079-42db-9f6d-fd53e986a411 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:12.091880 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 3/5] 10.222.0.42 () {56 vars in 1207 bytes} [Wed Jun 15 14:30:12 2022] GET /identity/v3/projects/688b80ae31764832aed9d6e0f3d3660f => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:14.188940 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a0f31d-4b1a-4de7-8795-83014cfdc00f None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:14.189490 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a0f31d-4b1a-4de7-8795-83014cfdc00f None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:14.189490 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a0f31d-4b1a-4de7-8795-83014cfdc00f None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:14.193651 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 3/6] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:30:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:14.199086 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4d5f04-60e8-4f82-af4b-fee1b8bd6a42 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:14.199488 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4d5f04-60e8-4f82-af4b-fee1b8bd6a42 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:14.199666 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4d5f04-60e8-4f82-af4b-fee1b8bd6a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:14.217947 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-3c4d5f04-60e8-4f82-af4b-fee1b8bd6a42 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:14.219552 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-3c4d5f04-60e8-4f82-af4b-fee1b8bd6a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:14.227306 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 4/7] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:14 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 15 14:30:14.250686 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a0f31d-4b1a-4de7-8795-83014cfdc00f None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:14.262691 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a0f31d-4b1a-4de7-8795-83014cfdc00f None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:14.267138 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0058472d-995b-4675-bc50-edb72480b379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:14.267765 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0058472d-995b-4675-bc50-edb72480b379 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:14.268052 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0058472d-995b-4675-bc50-edb72480b379 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:14.268365 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0058472d-995b-4675-bc50-edb72480b379 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:14.276227 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0058472d-995b-4675-bc50-edb72480b379 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:14.277993 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0058472d-995b-4675-bc50-edb72480b379 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:14.286680 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 4/8] 10.222.0.42 () {58 vars in 1324 bytes} [Wed Jun 15 14:30:14 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 15 14:30:14.292076 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4d5f04-60e8-4f82-af4b-fee1b8bd6a42 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:14.301806 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4d5f04-60e8-4f82-af4b-fee1b8bd6a42 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:14.306155 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49e1c55-3177-42ad-bea7-d453394a7d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:14.307006 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49e1c55-3177-42ad-bea7-d453394a7d06 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:14.307288 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49e1c55-3177-42ad-bea7-d453394a7d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:14.307586 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49e1c55-3177-42ad-bea7-d453394a7d06 admin admin] PATH_INFO: `/v3/users/11d932a20a0f4d8b9ce5ee15655d936c` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:14.308965 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49e1c55-3177-42ad-bea7-d453394a7d06 admin admin] RBAC: Authorizing `identity:get_user(user_id=11d932a20a0f4d8b9ce5ee15655d936c)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:14.310897 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49e1c55-3177-42ad-bea7-d453394a7d06 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:14.313260 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 5/9] 10.222.0.42 () {56 vars in 1198 bytes} [Wed Jun 15 14:30:14 2022] GET /identity/v3/users/11d932a20a0f4d8b9ce5ee15655d936c => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:16.505700 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923c3b7-4383-4e37-b9aa-004709519bd6 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:16.506230 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923c3b7-4383-4e37-b9aa-004709519bd6 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:16.506335 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923c3b7-4383-4e37-b9aa-004709519bd6 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:16.507627 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 5/10] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:30:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:30:16.512842 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb3d70-057d-492d-ada3-baec05616fd0 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:16.513217 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb3d70-057d-492d-ada3-baec05616fd0 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:16.513522 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb3d70-057d-492d-ada3-baec05616fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:16.521550 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07099fa8-fbde-4305-aa9c-f29429e4f55d None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:16.521834 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07099fa8-fbde-4305-aa9c-f29429e4f55d None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:16.522114 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07099fa8-fbde-4305-aa9c-f29429e4f55d None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:16.523443 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 6/11] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:30:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:30:16.528768 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae41f5-fec5-4646-859a-9bab889e0eee None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:16.529051 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae41f5-fec5-4646-859a-9bab889e0eee None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:16.529330 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae41f5-fec5-4646-859a-9bab889e0eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:16.532159 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-49eb3d70-057d-492d-ada3-baec05616fd0 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:16.533383 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-49eb3d70-057d-492d-ada3-baec05616fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:16.571038 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG passlib.handlers.bcrypt [None req-fcae41f5-fec5-4646-859a-9bab889e0eee None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80600) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 15 14:30:16.571458 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG passlib.handlers.bcrypt [None req-fcae41f5-fec5-4646-859a-9bab889e0eee None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80600) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 15 14:30:16.615473 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 6/12] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:30:16.617072 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-fcae41f5-fec5-4646-859a-9bab889e0eee None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:16.618685 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-fcae41f5-fec5-4646-859a-9bab889e0eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:16.638549 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb3d70-057d-492d-ada3-baec05616fd0 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:16.662223 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:16.721555 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 7/13] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:30:16.744227 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcae41f5-fec5-4646-859a-9bab889e0eee None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:16.745750 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb3d70-057d-492d-ada3-baec05616fd0 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:16.780332 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa22170-1177-49a4-b3f0-2be823906fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:16.781649 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa22170-1177-49a4-b3f0-2be823906fa3 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:16.781925 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa22170-1177-49a4-b3f0-2be823906fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:16.782198 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa22170-1177-49a4-b3f0-2be823906fa3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:16.793486 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa22170-1177-49a4-b3f0-2be823906fa3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:30:16.805358 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-3fa22170-1177-49a4-b3f0-2be823906fa3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:30:16.805358 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:16.805358 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:16.805358 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:16.805358 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:16.805358 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:16.805358 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:16.805358 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:16.805358 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:16.805358 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:16.805358 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:16.805358 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:16.805358 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:30:16.805358 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:30:16.805358 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:30:16.805358 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:30:16.806486 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:30:16.806486 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:30:16.806486 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:30:16.806486 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:30:16.806486 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:16.806486 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:16.806486 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:16.806486 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:16.806486 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:16.806486 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:16.806486 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:16.806486 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:16.806486 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:16.806486 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:16.806486 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:16.806486 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:16.807445 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:16.807445 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:16.807445 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:16.807445 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:16.807445 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:30:16.807445 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:30:16.807445 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:30:16.807445 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:30:16.807445 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:30:16.807445 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:30:16.807445 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:30:16.807445 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:30:16.807445 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 7/14] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jun 15 14:30:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:30:16.812486 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa22170-1177-49a4-b3f0-2be823906fa3 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:16.848735 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcae41f5-fec5-4646-859a-9bab889e0eee None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:16.850995 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:16.885597 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe3d461-9ee3-40a5-b757-4ed051a34053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:16.886488 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe3d461-9ee3-40a5-b757-4ed051a34053 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:16.886781 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe3d461-9ee3-40a5-b757-4ed051a34053 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:16.887063 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe3d461-9ee3-40a5-b757-4ed051a34053 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:16.893858 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:16.912841 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa22170-1177-49a4-b3f0-2be823906fa3 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:16.947416 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4409990-d35e-42df-a34d-f9471759cc88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:16.948201 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4409990-d35e-42df-a34d-f9471759cc88 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:16.948552 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4409990-d35e-42df-a34d-f9471759cc88 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:16.948837 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4409990-d35e-42df-a34d-f9471759cc88 None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:16.949588 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4409990-d35e-42df-a34d-f9471759cc88 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:16.949899 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4409990-d35e-42df-a34d-f9471759cc88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:16.951288 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4409990-d35e-42df-a34d-f9471759cc88 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:16.956610 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe3d461-9ee3-40a5-b757-4ed051a34053 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:16.958118 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe3d461-9ee3-40a5-b757-4ed051a34053 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:16.963029 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 8/15] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jun 15 14:30:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:16.968924 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4409990-d35e-42df-a34d-f9471759cc88 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:17.059293 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 8/16] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jun 15 14:30:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 15 14:30:17.065065 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe3d461-9ee3-40a5-b757-4ed051a34053 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:17.067510 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:17.067655 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4409990-d35e-42df-a34d-f9471759cc88 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:17.102119 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552a2e6d-2b60-469a-9c54-88eebb1a894a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:17.102991 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a2e6d-2b60-469a-9c54-88eebb1a894a None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:17.103275 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a2e6d-2b60-469a-9c54-88eebb1a894a None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:17.103554 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a2e6d-2b60-469a-9c54-88eebb1a894a None admin] PATH_INFO: `/v3/users/11d932a20a0f4d8b9ce5ee15655d936c` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:17.104790 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552a2e6d-2b60-469a-9c54-88eebb1a894a None admin] RBAC: Authorizing `identity:get_user(user_id=11d932a20a0f4d8b9ce5ee15655d936c)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:17.106219 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552a2e6d-2b60-469a-9c54-88eebb1a894a None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:17.107948 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 9/17] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jun 15 14:30:16 2022] GET /identity/v3/users/11d932a20a0f4d8b9ce5ee15655d936c => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:17.113249 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552a2e6d-2b60-469a-9c54-88eebb1a894a None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:17.127312 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:17.166430 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe3d461-9ee3-40a5-b757-4ed051a34053 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:17.202432 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad94e4f-9eb0-48b2-9ba6-16c066745d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:17.203570 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad94e4f-9eb0-48b2-9ba6-16c066745d0e None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:17.203856 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad94e4f-9eb0-48b2-9ba6-16c066745d0e None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:17.204172 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad94e4f-9eb0-48b2-9ba6-16c066745d0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:17.213991 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552a2e6d-2b60-469a-9c54-88eebb1a894a None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:17.222091 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad94e4f-9eb0-48b2-9ba6-16c066745d0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:17.223540 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad94e4f-9eb0-48b2-9ba6-16c066745d0e None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:17.244347 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-9ad94e4f-9eb0-48b2-9ba6-16c066745d0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:17.244347 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:17.244347 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:17.244347 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:17.244347 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:17.244347 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:17.244347 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:17.244347 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:17.244347 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:17.244347 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:17.244347 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:17.244347 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:17.244347 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:17.244347 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:17.244347 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:17.245387 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:17.245387 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:17.245387 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:17.245387 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:17.245387 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:17.245387 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:17.245387 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:17.245387 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:17.245387 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:17.245387 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:17.245387 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:17.245387 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:17.245387 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:17.245387 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:17.245387 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:17.245387 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:17.245387 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:17.246245 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:17.246245 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:17.246245 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:17.246245 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:30:17.246245 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 9/18] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:30:17.252575 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad94e4f-9eb0-48b2-9ba6-16c066745d0e None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:17.253596 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aaf6bf-820c-4092-b588-6a6e11ca386d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:17.254393 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aaf6bf-820c-4092-b588-6a6e11ca386d None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:17.254679 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aaf6bf-820c-4092-b588-6a6e11ca386d None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:17.254960 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aaf6bf-820c-4092-b588-6a6e11ca386d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:17.256168 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aaf6bf-820c-4092-b588-6a6e11ca386d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:17.257632 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aaf6bf-820c-4092-b588-6a6e11ca386d None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:17.259226 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 10/19] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:17.264893 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aaf6bf-820c-4092-b588-6a6e11ca386d None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:17.307958 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:17.348353 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:17.350625 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad94e4f-9eb0-48b2-9ba6-16c066745d0e None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:17.367984 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aaf6bf-820c-4092-b588-6a6e11ca386d None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:17.385082 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d766d6f6-eba4-417c-a256-11cf2623620f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:17.385725 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d766d6f6-eba4-417c-a256-11cf2623620f None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:17.386010 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d766d6f6-eba4-417c-a256-11cf2623620f None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:17.386290 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d766d6f6-eba4-417c-a256-11cf2623620f None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:17.387018 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d766d6f6-eba4-417c-a256-11cf2623620f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:17.387334 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d766d6f6-eba4-417c-a256-11cf2623620f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:17.388728 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d766d6f6-eba4-417c-a256-11cf2623620f None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:17.402863 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dd74fc-c1f7-423b-a629-6ef9adff1111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:17.403520 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dd74fc-c1f7-423b-a629-6ef9adff1111 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:17.403801 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dd74fc-c1f7-423b-a629-6ef9adff1111 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:17.404078 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dd74fc-c1f7-423b-a629-6ef9adff1111 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:17.404941 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dd74fc-c1f7-423b-a629-6ef9adff1111 None admin] RBAC: Adding query filter params (role.id=2bd1979f53bb4853a50dd5bee6aa0782, scope.domain.id=default, user.id=11d932a20a0f4d8b9ce5ee15655d936c) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:17.405258 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dd74fc-c1f7-423b-a629-6ef9adff1111 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:17.407004 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 10/20] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:17.407559 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dd74fc-c1f7-423b-a629-6ef9adff1111 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:17.414712 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 11/21] 10.222.0.42 () {56 vars in 1322 bytes} [Wed Jun 15 14:30:17 2022] GET /identity/v3/role_assignments?user.id=11d932a20a0f4d8b9ce5ee15655d936c&scope.domain.id=default&role.id=2bd1979f53bb4853a50dd5bee6aa0782 => generated 233 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:20.048027 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae9431-4116-4461-bd50-c02b13601264 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:20.048027 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae9431-4116-4461-bd50-c02b13601264 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:20.048027 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae9431-4116-4461-bd50-c02b13601264 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:20.048027 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 11/22] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:30:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:20.048809 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ad30cb-6725-4eaf-99ce-2b0b94ce5934 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:20.048809 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ad30cb-6725-4eaf-99ce-2b0b94ce5934 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:20.048809 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ad30cb-6725-4eaf-99ce-2b0b94ce5934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:20.048809 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-d2ad30cb-6725-4eaf-99ce-2b0b94ce5934 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:20.048809 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-d2ad30cb-6725-4eaf-99ce-2b0b94ce5934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:20.077779 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201bed08-094d-4feb-a645-110536612d03 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:20.078187 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201bed08-094d-4feb-a645-110536612d03 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:20.078442 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201bed08-094d-4feb-a645-110536612d03 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:20.079770 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 12/23] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:30:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:30:20.086494 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7707a6b9-3038-4570-b5d5-c3effe9b7a12 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:20.086780 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7707a6b9-3038-4570-b5d5-c3effe9b7a12 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:20.087058 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7707a6b9-3038-4570-b5d5-c3effe9b7a12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:20.149169 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-7707a6b9-3038-4570-b5d5-c3effe9b7a12 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:20.150398 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-7707a6b9-3038-4570-b5d5-c3effe9b7a12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:20.193596 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 12/24] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:30:20.200303 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49a02c3-50f4-419c-860f-e17ecbe8594b None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:20.200906 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49a02c3-50f4-419c-860f-e17ecbe8594b None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:20.201389 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49a02c3-50f4-419c-860f-e17ecbe8594b None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:20.203067 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 13/25] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:30:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:30:20.208409 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae6ca1f-8f43-4cf2-a995-ffb4ee922a25 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:20.208927 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae6ca1f-8f43-4cf2-a995-ffb4ee922a25 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:20.209424 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae6ca1f-8f43-4cf2-a995-ffb4ee922a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:20.240258 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-2ae6ca1f-8f43-4cf2-a995-ffb4ee922a25 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:20.241810 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-2ae6ca1f-8f43-4cf2-a995-ffb4ee922a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:20.337376 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 13/26] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:30:20.339575 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7707a6b9-3038-4570-b5d5-c3effe9b7a12 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:20.342760 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 14/27] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:30:20.385170 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae6ca1f-8f43-4cf2-a995-ffb4ee922a25 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:20.396422 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:20.452658 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:20.559876 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7707a6b9-3038-4570-b5d5-c3effe9b7a12 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:20.599557 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac0346b-163a-4baa-b630-814f5948ba16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:20.600514 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac0346b-163a-4baa-b630-814f5948ba16 None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:20.600957 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac0346b-163a-4baa-b630-814f5948ba16 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:20.601376 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac0346b-163a-4baa-b630-814f5948ba16 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:20.602269 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac0346b-163a-4baa-b630-814f5948ba16 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:20.603822 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac0346b-163a-4baa-b630-814f5948ba16 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:20.606408 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae6ca1f-8f43-4cf2-a995-ffb4ee922a25 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:20.643284 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b009218-6f0c-4e0a-8428-12ea5ccf9db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:20.644348 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b009218-6f0c-4e0a-8428-12ea5ccf9db1 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:20.644796 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b009218-6f0c-4e0a-8428-12ea5ccf9db1 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:20.645232 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b009218-6f0c-4e0a-8428-12ea5ccf9db1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:20.657433 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b009218-6f0c-4e0a-8428-12ea5ccf9db1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:30:20.665503 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-4b009218-6f0c-4e0a-8428-12ea5ccf9db1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:30:20.665503 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:20.665503 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:20.665503 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:20.665503 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:20.665503 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:20.665503 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:20.665503 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:20.665503 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:20.665503 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:20.665503 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:20.665503 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:20.665503 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:30:20.665503 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:30:20.665503 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:30:20.665503 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:30:20.666574 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:30:20.666574 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:30:20.666574 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:30:20.666574 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:30:20.666574 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:20.666574 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:20.666574 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:20.666574 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:20.666574 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:20.666574 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:20.666574 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:20.666574 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:20.666574 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:20.666574 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:20.666574 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:20.666574 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:20.667602 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:20.667602 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:20.667602 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:20.667602 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:20.667602 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:30:20.667602 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:30:20.667602 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:30:20.667602 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:30:20.667602 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:30:20.667602 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:30:20.667602 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:30:20.667602 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:30:20.667602 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 15/28] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jun 15 14:30:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:20.667602 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b009218-6f0c-4e0a-8428-12ea5ccf9db1 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:20.741823 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:20.786082 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b009218-6f0c-4e0a-8428-12ea5ccf9db1 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:20.836113 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806287e0-254c-4752-9f8e-71e422dce2d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:20.836113 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806287e0-254c-4752-9f8e-71e422dce2d5 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:20.836113 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806287e0-254c-4752-9f8e-71e422dce2d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:20.836113 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806287e0-254c-4752-9f8e-71e422dce2d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:20.869440 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806287e0-254c-4752-9f8e-71e422dce2d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:20.870886 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806287e0-254c-4752-9f8e-71e422dce2d5 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:20.893024 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-806287e0-254c-4752-9f8e-71e422dce2d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:20.893024 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:20.893024 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:20.893024 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:20.893024 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:20.893024 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:20.893024 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:20.893024 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:20.893024 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:20.893024 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:20.893024 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:20.893024 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:20.893024 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:20.893024 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:20.893024 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:20.893924 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:20.893924 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:20.893924 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:20.893924 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:20.893924 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:20.893924 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:20.893924 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:20.893924 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:20.893924 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:20.893924 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:20.893924 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:20.893924 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:20.893924 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:20.893924 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:20.893924 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:20.893924 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:20.893924 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:20.894734 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:20.894734 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:20.894734 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:20.894734 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:30:20.894734 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 16/29] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:20.895454 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806287e0-254c-4752-9f8e-71e422dce2d5 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:20.925913 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 14/30] 10.222.0.42 () {60 vars in 1131 bytes} [Wed Jun 15 14:30:20 2022] POST /identity/v3/roles => generated 219 bytes in 586 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:30:20.926795 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac0346b-163a-4baa-b630-814f5948ba16 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:21.024802 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806287e0-254c-4752-9f8e-71e422dce2d5 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:21.052689 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:21.068739 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d6852c-cfdd-4d6d-8f34-43eaa652ea38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:21.069425 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d6852c-cfdd-4d6d-8f34-43eaa652ea38 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:21.069709 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d6852c-cfdd-4d6d-8f34-43eaa652ea38 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:21.069984 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d6852c-cfdd-4d6d-8f34-43eaa652ea38 None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:21.070746 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d6852c-cfdd-4d6d-8f34-43eaa652ea38 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:21.071054 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d6852c-cfdd-4d6d-8f34-43eaa652ea38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:21.077046 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d6852c-cfdd-4d6d-8f34-43eaa652ea38 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:21.094406 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 17/31] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jun 15 14:30:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:21.095624 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b7c0df-56d0-4f81-8486-ef346d0ac13a None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:21.095937 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b7c0df-56d0-4f81-8486-ef346d0ac13a None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:21.096232 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b7c0df-56d0-4f81-8486-ef346d0ac13a None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:21.106904 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 18/32] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:30:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:21.109802 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc90ca-637b-4a69-b83c-17a6e2285973 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:21.110098 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc90ca-637b-4a69-b83c-17a6e2285973 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:21.110379 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc90ca-637b-4a69-b83c-17a6e2285973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:21.146236 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-16cc90ca-637b-4a69-b83c-17a6e2285973 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:21.147367 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-16cc90ca-637b-4a69-b83c-17a6e2285973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:21.193689 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac0346b-163a-4baa-b630-814f5948ba16 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:21.203295 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:21.243901 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20af2bfb-290a-4635-9dc7-5a70b9003d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:21.244626 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20af2bfb-290a-4635-9dc7-5a70b9003d48 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:21.244911 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20af2bfb-290a-4635-9dc7-5a70b9003d48 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:21.245192 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20af2bfb-290a-4635-9dc7-5a70b9003d48 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:21.245932 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20af2bfb-290a-4635-9dc7-5a70b9003d48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:21.246246 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20af2bfb-290a-4635-9dc7-5a70b9003d48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:21.247753 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20af2bfb-290a-4635-9dc7-5a70b9003d48 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:21.256316 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 19/33] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:30:21.257570 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cc90ca-637b-4a69-b83c-17a6e2285973 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:21.269041 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 15/34] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:21.270858 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20af2bfb-290a-4635-9dc7-5a70b9003d48 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:21.396733 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cc90ca-637b-4a69-b83c-17a6e2285973 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:21.406755 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:21.418663 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20af2bfb-290a-4635-9dc7-5a70b9003d48 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:21.432094 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39292a58-6507-4aa0-a809-29ee801235a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:21.432983 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39292a58-6507-4aa0-a809-29ee801235a6 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:21.433270 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39292a58-6507-4aa0-a809-29ee801235a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:21.433658 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39292a58-6507-4aa0-a809-29ee801235a6 None admin] PATH_INFO: `/v3/users/11d932a20a0f4d8b9ce5ee15655d936c` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:21.434867 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39292a58-6507-4aa0-a809-29ee801235a6 None admin] RBAC: Authorizing `identity:get_user(user_id=11d932a20a0f4d8b9ce5ee15655d936c)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:21.436355 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39292a58-6507-4aa0-a809-29ee801235a6 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:21.438362 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 20/35] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jun 15 14:30:21 2022] GET /identity/v3/users/11d932a20a0f4d8b9ce5ee15655d936c => generated 247 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:21.439183 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39292a58-6507-4aa0-a809-29ee801235a6 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:21.452453 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91191597-3ddf-4bd3-80f4-d8c7e9391ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:21.453172 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91191597-3ddf-4bd3-80f4-d8c7e9391ebe None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:21.453569 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91191597-3ddf-4bd3-80f4-d8c7e9391ebe None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:21.453860 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91191597-3ddf-4bd3-80f4-d8c7e9391ebe None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:21.456235 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91191597-3ddf-4bd3-80f4-d8c7e9391ebe None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:21.456348 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91191597-3ddf-4bd3-80f4-d8c7e9391ebe None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:21.545031 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39292a58-6507-4aa0-a809-29ee801235a6 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:21.573083 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 16/36] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Jun 15 14:30:21 2022] POST /identity/v3/projects => generated 301 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:30:21.578632 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b07e18-c4f2-42c8-a969-6127e5ea828c None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:21.578632 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b07e18-c4f2-42c8-a969-6127e5ea828c None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:21.578632 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b07e18-c4f2-42c8-a969-6127e5ea828c None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:21.578632 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 17/37] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:30:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:21.578632 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacf8322-1ea9-492d-a08f-cbb741b072dd None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:21.578632 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacf8322-1ea9-492d-a08f-cbb741b072dd None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:21.579135 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacf8322-1ea9-492d-a08f-cbb741b072dd None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:21.584640 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 18/38] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:30:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:21.584640 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacf8322-1ea9-492d-a08f-cbb741b072dd None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:21.594119 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c18b5c-98d9-4b9c-94e2-5a06dfd2fe9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:21.594823 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c18b5c-98d9-4b9c-94e2-5a06dfd2fe9e None admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:21.595104 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c18b5c-98d9-4b9c-94e2-5a06dfd2fe9e None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:21.595383 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c18b5c-98d9-4b9c-94e2-5a06dfd2fe9e None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:21.596210 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c18b5c-98d9-4b9c-94e2-5a06dfd2fe9e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:21.597651 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c18b5c-98d9-4b9c-94e2-5a06dfd2fe9e None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:21.605281 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:21.695121 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacf8322-1ea9-492d-a08f-cbb741b072dd None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:21.737153 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3b65ab-5921-443f-9a77-f60619a08d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:21.738032 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3b65ab-5921-443f-9a77-f60619a08d3f None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:21.738329 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3b65ab-5921-443f-9a77-f60619a08d3f None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:21.738612 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3b65ab-5921-443f-9a77-f60619a08d3f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:21.739993 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3b65ab-5921-443f-9a77-f60619a08d3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:21.741740 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3b65ab-5921-443f-9a77-f60619a08d3f None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:21.743685 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 19/39] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:21.746597 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e025556-2328-4417-a196-0a182f3daf19 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:21.746597 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e025556-2328-4417-a196-0a182f3daf19 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:21.746597 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e025556-2328-4417-a196-0a182f3daf19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:21.767719 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-0e025556-2328-4417-a196-0a182f3daf19 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:21.769919 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-0e025556-2328-4417-a196-0a182f3daf19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:21.849415 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 20/40] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:30:21.850495 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd57b62-3503-46fc-adf7-e0cf459c66aa None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:21.850786 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd57b62-3503-46fc-adf7-e0cf459c66aa None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:21.851067 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd57b62-3503-46fc-adf7-e0cf459c66aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:21.869048 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-1bd57b62-3503-46fc-adf7-e0cf459c66aa None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:21.870229 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-1bd57b62-3503-46fc-adf7-e0cf459c66aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:21.882577 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:21.916797 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 21/41] 10.222.0.42 () {60 vars in 1131 bytes} [Wed Jun 15 14:30:21 2022] POST /identity/v3/roles => generated 225 bytes in 478 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:30:21.917748 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c18b5c-98d9-4b9c-94e2-5a06dfd2fe9e None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:21.956621 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 21/42] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:30:21.957255 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd57b62-3503-46fc-adf7-e0cf459c66aa None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:21.965324 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:22.043268 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c18b5c-98d9-4b9c-94e2-5a06dfd2fe9e None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:22.085247 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fdbb2e-e2b0-4293-8703-2ab6ae28d0f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:22.086151 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fdbb2e-e2b0-4293-8703-2ab6ae28d0f6 None admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:22.086435 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fdbb2e-e2b0-4293-8703-2ab6ae28d0f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:22.086744 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fdbb2e-e2b0-4293-8703-2ab6ae28d0f6 None admin] PATH_INFO: `/v3/domains/default/users/11d932a20a0f4d8b9ce5ee15655d936c/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:22.089075 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fdbb2e-e2b0-4293-8703-2ab6ae28d0f6 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=11d932a20a0f4d8b9ce5ee15655d936c, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:22.091061 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fdbb2e-e2b0-4293-8703-2ab6ae28d0f6 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:22.182288 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 22/43] 10.222.0.42 () {58 vars in 1361 bytes} [Wed Jun 15 14:30:21 2022] PUT /identity/v3/domains/default/users/11d932a20a0f4d8b9ce5ee15655d936c/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:30:22.182760 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd57b62-3503-46fc-adf7-e0cf459c66aa None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:22.183172 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fdbb2e-e2b0-4293-8703-2ab6ae28d0f6 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:22.233041 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f0103a-7025-4291-a00d-07a3a84796c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:22.233569 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f0103a-7025-4291-a00d-07a3a84796c1 None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:22.233960 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f0103a-7025-4291-a00d-07a3a84796c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:22.234921 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f0103a-7025-4291-a00d-07a3a84796c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:22.235702 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f0103a-7025-4291-a00d-07a3a84796c1 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:22.237183 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f0103a-7025-4291-a00d-07a3a84796c1 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:22.293945 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:22.299935 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fdbb2e-e2b0-4293-8703-2ab6ae28d0f6 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:22.328157 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 22/44] 10.222.0.42 () {60 vars in 1131 bytes} [Wed Jun 15 14:30:21 2022] POST /identity/v3/roles => generated 223 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:30:22.341134 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a50e49-ca2d-4dd2-862d-ef4c0797f011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:22.341961 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a50e49-ca2d-4dd2-862d-ef4c0797f011 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:22.342312 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a50e49-ca2d-4dd2-862d-ef4c0797f011 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:22.342591 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a50e49-ca2d-4dd2-862d-ef4c0797f011 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:22.344476 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a50e49-ca2d-4dd2-862d-ef4c0797f011 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:22.346118 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a50e49-ca2d-4dd2-862d-ef4c0797f011 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:22.347852 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 23/45] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:22.353852 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f0103a-7025-4291-a00d-07a3a84796c1 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:22.363337 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:22.473875 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f0103a-7025-4291-a00d-07a3a84796c1 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:22.511781 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f4cfdb-430b-4340-bf24-6dddd54de7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:22.512488 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f4cfdb-430b-4340-bf24-6dddd54de7bf None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:22.512779 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f4cfdb-430b-4340-bf24-6dddd54de7bf None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:22.513062 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f4cfdb-430b-4340-bf24-6dddd54de7bf None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:22.513783 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f4cfdb-430b-4340-bf24-6dddd54de7bf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:22.515337 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f4cfdb-430b-4340-bf24-6dddd54de7bf None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:22.595365 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 23/46] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Jun 15 14:30:22 2022] POST /identity/v3/projects => generated 312 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:30:24.524501 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1c24b4-1c3c-4a2e-984b-ff67c196144d None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:24.525045 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1c24b4-1c3c-4a2e-984b-ff67c196144d None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:24.525045 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1c24b4-1c3c-4a2e-984b-ff67c196144d None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:24.526822 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 24/47] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:30:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:24.531871 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5d5357-de46-4469-a763-25829ce2e405 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:24.532331 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5d5357-de46-4469-a763-25829ce2e405 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:24.532459 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5d5357-de46-4469-a763-25829ce2e405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:24.549929 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-9c5d5357-de46-4469-a763-25829ce2e405 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:24.551135 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-9c5d5357-de46-4469-a763-25829ce2e405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:24.606932 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:24.620472 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 24/48] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:30:24.643876 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1c24b4-1c3c-4a2e-984b-ff67c196144d None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:24.750623 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1c24b4-1c3c-4a2e-984b-ff67c196144d None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:24.760661 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3aed33-788f-448c-8475-261ad4dfc654 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:24.760898 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3aed33-788f-448c-8475-261ad4dfc654 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:24.761178 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3aed33-788f-448c-8475-261ad4dfc654 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:24.762581 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 25/49] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:30:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:30:24.767632 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf92b21-1995-4312-b031-e3a825944eca None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:24.767925 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf92b21-1995-4312-b031-e3a825944eca None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:24.768254 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf92b21-1995-4312-b031-e3a825944eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:24.784718 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-bdf92b21-1995-4312-b031-e3a825944eca None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:24.785846 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-bdf92b21-1995-4312-b031-e3a825944eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:24.787297 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c8dc15-7b3b-40e6-aa45-752f0d495238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:24.788082 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c8dc15-7b3b-40e6-aa45-752f0d495238 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:24.788395 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c8dc15-7b3b-40e6-aa45-752f0d495238 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:24.788676 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c8dc15-7b3b-40e6-aa45-752f0d495238 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:24.799123 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c8dc15-7b3b-40e6-aa45-752f0d495238 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:30:24.807029 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-e9c8dc15-7b3b-40e6-aa45-752f0d495238 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:30:24.807029 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:24.807029 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:24.807029 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:24.807029 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:24.807029 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:24.807029 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:24.807029 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:24.807029 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:24.807029 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:24.807029 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:24.807029 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:24.807029 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:30:24.807029 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:30:24.807029 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:30:24.807029 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:30:24.808050 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:30:24.808050 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:30:24.808050 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:30:24.808050 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:30:24.808050 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:24.808050 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:24.808050 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:24.808050 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:24.808050 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:24.808050 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:24.808050 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:24.808050 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:24.808050 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:24.808050 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:24.808050 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:24.808050 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:24.809029 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:24.809029 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:24.809029 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:24.809029 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:24.809029 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:30:24.809029 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:30:24.809029 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:30:24.809029 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:30:24.809029 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:30:24.809029 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:30:24.809029 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:30:24.809029 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:30:24.809029 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 25/50] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jun 15 14:30:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:24.816293 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c8dc15-7b3b-40e6-aa45-752f0d495238 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:24.841843 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:24.855783 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 26/51] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:30:24.879928 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf92b21-1995-4312-b031-e3a825944eca None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:24.923434 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c8dc15-7b3b-40e6-aa45-752f0d495238 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:24.957339 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98495008-3ed9-47cf-94d7-794ea1ddd650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:24.958006 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98495008-3ed9-47cf-94d7-794ea1ddd650 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:24.958286 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98495008-3ed9-47cf-94d7-794ea1ddd650 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:24.958563 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98495008-3ed9-47cf-94d7-794ea1ddd650 None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:24.959306 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98495008-3ed9-47cf-94d7-794ea1ddd650 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:24.959613 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98495008-3ed9-47cf-94d7-794ea1ddd650 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:24.961019 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98495008-3ed9-47cf-94d7-794ea1ddd650 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:24.969989 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 26/52] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jun 15 14:30:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:24.975439 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98495008-3ed9-47cf-94d7-794ea1ddd650 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:24.982214 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf92b21-1995-4312-b031-e3a825944eca None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:25.000739 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:25.019235 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c904dc-c578-40f3-9326-d86fc2227d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:25.020036 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c904dc-c578-40f3-9326-d86fc2227d78 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:25.020355 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c904dc-c578-40f3-9326-d86fc2227d78 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:25.020640 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c904dc-c578-40f3-9326-d86fc2227d78 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:25.021857 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c904dc-c578-40f3-9326-d86fc2227d78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:25.023333 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c904dc-c578-40f3-9326-d86fc2227d78 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:25.024992 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 27/53] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:25.030582 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c904dc-c578-40f3-9326-d86fc2227d78 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:25.063378 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:25.079011 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98495008-3ed9-47cf-94d7-794ea1ddd650 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:25.112875 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6739ba-0d95-4982-8ba3-6787d75832cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:25.113763 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6739ba-0d95-4982-8ba3-6787d75832cf None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:25.114053 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6739ba-0d95-4982-8ba3-6787d75832cf None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:25.114333 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6739ba-0d95-4982-8ba3-6787d75832cf None admin] PATH_INFO: `/v3/users/11d932a20a0f4d8b9ce5ee15655d936c` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:25.115541 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6739ba-0d95-4982-8ba3-6787d75832cf None admin] RBAC: Authorizing `identity:get_user(user_id=11d932a20a0f4d8b9ce5ee15655d936c)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:25.117210 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6739ba-0d95-4982-8ba3-6787d75832cf None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:25.119063 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 27/54] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jun 15 14:30:24 2022] GET /identity/v3/users/11d932a20a0f4d8b9ce5ee15655d936c => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:25.124927 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6739ba-0d95-4982-8ba3-6787d75832cf None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:25.139543 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c904dc-c578-40f3-9326-d86fc2227d78 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:25.174040 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5182b447-bbe1-406b-8222-902c92956c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:25.174695 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5182b447-bbe1-406b-8222-902c92956c22 None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:25.174980 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5182b447-bbe1-406b-8222-902c92956c22 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:25.175266 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5182b447-bbe1-406b-8222-902c92956c22 None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:25.175978 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5182b447-bbe1-406b-8222-902c92956c22 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:25.177443 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5182b447-bbe1-406b-8222-902c92956c22 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:25.225754 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6739ba-0d95-4982-8ba3-6787d75832cf None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:25.258681 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453b476e-ec99-49b5-befe-19bf757d6e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:25.259477 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b476e-ec99-49b5-befe-19bf757d6e3d None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:25.259769 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b476e-ec99-49b5-befe-19bf757d6e3d None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:25.260056 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b476e-ec99-49b5-befe-19bf757d6e3d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:25.261296 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453b476e-ec99-49b5-befe-19bf757d6e3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:25.262710 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453b476e-ec99-49b5-befe-19bf757d6e3d None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:25.264363 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 28/55] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:25.270756 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453b476e-ec99-49b5-befe-19bf757d6e3d None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:25.294331 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:25.316462 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 28/56] 10.222.0.42 () {60 vars in 1140 bytes} [Wed Jun 15 14:30:25 2022] POST /identity/v3/projects => generated 298 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:30:25.375174 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453b476e-ec99-49b5-befe-19bf757d6e3d None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:25.412487 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a3899a-7712-4651-a355-3163360e4a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:25.413165 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a3899a-7712-4651-a355-3163360e4a69 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:25.413456 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a3899a-7712-4651-a355-3163360e4a69 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:25.413744 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a3899a-7712-4651-a355-3163360e4a69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:25.414603 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a3899a-7712-4651-a355-3163360e4a69 None admin] RBAC: Adding query filter params (role.id=2bd1979f53bb4853a50dd5bee6aa0782, scope.domain.id=default, user.id=11d932a20a0f4d8b9ce5ee15655d936c) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:25.414925 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a3899a-7712-4651-a355-3163360e4a69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:25.416439 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a3899a-7712-4651-a355-3163360e4a69 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:25.423679 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 29/57] 10.222.0.42 () {56 vars in 1322 bytes} [Wed Jun 15 14:30:25 2022] GET /identity/v3/role_assignments?user.id=11d932a20a0f4d8b9ce5ee15655d936c&scope.domain.id=default&role.id=2bd1979f53bb4853a50dd5bee6aa0782 => generated 530 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:27.440033 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca48415-8b9f-4acb-8d16-f4605f465f21 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:27.440560 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca48415-8b9f-4acb-8d16-f4605f465f21 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:27.440642 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca48415-8b9f-4acb-8d16-f4605f465f21 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:27.441989 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 29/58] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:30:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:27.447174 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8310a7-1afb-4a79-8f6b-f8af11efb762 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:27.447552 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8310a7-1afb-4a79-8f6b-f8af11efb762 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:27.447749 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8310a7-1afb-4a79-8f6b-f8af11efb762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:27.464567 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-fe8310a7-1afb-4a79-8f6b-f8af11efb762 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:27.466111 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-fe8310a7-1afb-4a79-8f6b-f8af11efb762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:27.532836 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 30/59] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:30:27.556958 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca48415-8b9f-4acb-8d16-f4605f465f21 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:27.597120 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:27.654690 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca48415-8b9f-4acb-8d16-f4605f465f21 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:27.687274 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84602ee3-4856-41e9-9c29-17f833e59268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:27.688054 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84602ee3-4856-41e9-9c29-17f833e59268 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:27.688372 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84602ee3-4856-41e9-9c29-17f833e59268 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:27.688647 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84602ee3-4856-41e9-9c29-17f833e59268 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:27.689809 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84602ee3-4856-41e9-9c29-17f833e59268 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:27.691270 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84602ee3-4856-41e9-9c29-17f833e59268 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:27.692857 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 30/60] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:27.698386 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8310a7-1afb-4a79-8f6b-f8af11efb762 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:27.716479 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:27.800879 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8310a7-1afb-4a79-8f6b-f8af11efb762 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:27.833539 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4768be7d-9790-4ae8-af33-07e1a27bdbfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:27.834199 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4768be7d-9790-4ae8-af33-07e1a27bdbfd None admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:27.834484 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4768be7d-9790-4ae8-af33-07e1a27bdbfd None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:27.834761 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4768be7d-9790-4ae8-af33-07e1a27bdbfd None admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:27.835469 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4768be7d-9790-4ae8-af33-07e1a27bdbfd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:27.836942 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4768be7d-9790-4ae8-af33-07e1a27bdbfd None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:27.931350 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 31/61] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Jun 15 14:30:27 2022] POST /identity/v3/users => generated 275 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:30:31.643302 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c34b44a-bb8b-4a8a-8b49-fd898cc7cb45 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:31.643791 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c34b44a-bb8b-4a8a-8b49-fd898cc7cb45 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:31.643884 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c34b44a-bb8b-4a8a-8b49-fd898cc7cb45 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:31.645587 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 31/62] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:30:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:31.651089 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9558d9-c161-4f74-b260-97c4b17755e4 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:31.651534 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9558d9-c161-4f74-b260-97c4b17755e4 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:31.651716 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9558d9-c161-4f74-b260-97c4b17755e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:31.681380 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-3e9558d9-c161-4f74-b260-97c4b17755e4 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:31.682603 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-3e9558d9-c161-4f74-b260-97c4b17755e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:31.787872 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a294434-1d95-4330-8cc0-3d1db3ba8ff5 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:31.788509 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a294434-1d95-4330-8cc0-3d1db3ba8ff5 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:31.788988 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a294434-1d95-4330-8cc0-3d1db3ba8ff5 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:31.790579 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 32/63] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:30:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:30:31.791806 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 32/64] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:30:31.798124 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051997f8-f2a7-4fe8-832e-f8a5b68242f1 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:31.798680 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051997f8-f2a7-4fe8-832e-f8a5b68242f1 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:31.799156 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051997f8-f2a7-4fe8-832e-f8a5b68242f1 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:31.803005 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a52c7d-3651-4de0-890f-035cf73c1c79 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:31.803242 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a52c7d-3651-4de0-890f-035cf73c1c79 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:31.803526 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a52c7d-3651-4de0-890f-035cf73c1c79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:31.804224 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 33/65] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:30:31 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:31.810798 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0396eee-532f-45d4-860e-90b72fa1dd0a None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:31.811326 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0396eee-532f-45d4-860e-90b72fa1dd0a None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:31.812223 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0396eee-532f-45d4-860e-90b72fa1dd0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:31.812527 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:31.825015 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-01a52c7d-3651-4de0-890f-035cf73c1c79 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:31.826546 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-01a52c7d-3651-4de0-890f-035cf73c1c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:31.855015 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-d0396eee-532f-45d4-860e-90b72fa1dd0a None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:31.858954 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-d0396eee-532f-45d4-860e-90b72fa1dd0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:31.909732 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 33/66] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:30:31.910866 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a52c7d-3651-4de0-890f-035cf73c1c79 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:31.948681 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 34/67] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:30:31.950008 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0396eee-532f-45d4-860e-90b72fa1dd0a None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:31.991122 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:32.068837 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0396eee-532f-45d4-860e-90b72fa1dd0a None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:32.071231 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a52c7d-3651-4de0-890f-035cf73c1c79 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:32.109422 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:32.118269 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8129df0e-7f48-4094-bf57-5fe4f68a402a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:32.124855 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fb3f9e-657a-4164-be77-b098d3498a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:32.124855 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb3f9e-657a-4164-be77-b098d3498a88 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:32.125229 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8129df0e-7f48-4094-bf57-5fe4f68a402a None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:32.125368 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb3f9e-657a-4164-be77-b098d3498a88 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:32.125468 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8129df0e-7f48-4094-bf57-5fe4f68a402a None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:32.125536 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb3f9e-657a-4164-be77-b098d3498a88 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:32.125612 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8129df0e-7f48-4094-bf57-5fe4f68a402a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:32.133419 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fb3f9e-657a-4164-be77-b098d3498a88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:30:32.133775 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8129df0e-7f48-4094-bf57-5fe4f68a402a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:30:32.143358 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-8129df0e-7f48-4094-bf57-5fe4f68a402a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:30:32.143358 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:32.143358 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:32.143358 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:32.143358 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:32.143358 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:32.143358 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:32.143358 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:32.143358 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:32.143358 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:32.143358 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:32.143358 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:32.143358 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:30:32.143358 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:30:32.143358 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:30:32.143358 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:30:32.144435 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:30:32.144435 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:30:32.144435 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:30:32.144435 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:30:32.144435 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:32.144435 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:32.144435 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:32.144435 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:32.144435 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:32.144435 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:32.144435 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:32.144435 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:32.144435 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:32.144435 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:32.144435 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:32.144435 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:32.145207 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:32.145207 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:32.145207 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:32.145207 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:32.145207 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:30:32.145207 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:30:32.145207 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:30:32.145207 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:30:32.145207 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:30:32.145207 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:30:32.145207 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:30:32.145207 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:30:32.145784 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-53fb3f9e-657a-4164-be77-b098d3498a88 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:30:32.145784 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:32.145784 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:32.145784 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:32.145784 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:32.145784 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:32.145784 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:32.145784 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:32.145784 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:32.145784 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:32.145784 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:32.145784 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:32.145784 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:30:32.145784 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:30:32.145784 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:30:32.145784 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:30:32.146571 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:30:32.146571 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:30:32.146571 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:30:32.146571 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:30:32.146571 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:32.146571 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:32.146571 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:32.146571 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:32.146571 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:32.146571 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:32.146571 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:32.146571 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:32.146571 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:32.146571 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:32.146571 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:32.146571 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:32.147363 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:32.147363 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:32.147363 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:32.147363 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:32.147363 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:30:32.147363 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:30:32.147363 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:30:32.147363 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:30:32.147363 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:30:32.147363 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:30:32.147363 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:30:32.147363 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:30:32.147363 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 34/69] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Jun 15 14:30:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:32.147363 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fb3f9e-657a-4164-be77-b098d3498a88 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:32.148159 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 35/69] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jun 15 14:30:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:32.150933 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b44b24-59d3-48d1-baf1-d3718447ea05 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:32.150933 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b44b24-59d3-48d1-baf1-d3718447ea05 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:32.150933 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b44b24-59d3-48d1-baf1-d3718447ea05 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:32.153128 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 36/70] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:30:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:32.153128 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b44b24-59d3-48d1-baf1-d3718447ea05 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:32.279020 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:32.288863 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b44b24-59d3-48d1-baf1-d3718447ea05 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:32.302453 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fb3f9e-657a-4164-be77-b098d3498a88 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:32.337724 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c695be85-d70e-4a55-83a7-4baf73e4e56a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:32.338601 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c695be85-d70e-4a55-83a7-4baf73e4e56a None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:32.339054 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c695be85-d70e-4a55-83a7-4baf73e4e56a None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:32.339340 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c695be85-d70e-4a55-83a7-4baf73e4e56a None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:32.339657 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3b6a16-d802-496d-bd19-b4f0a9c5f42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:32.340133 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c695be85-d70e-4a55-83a7-4baf73e4e56a None admin] RBAC: Adding query filter params (name=member) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:32.340493 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3b6a16-d802-496d-bd19-b4f0a9c5f42b None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:32.340610 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c695be85-d70e-4a55-83a7-4baf73e4e56a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:32.341014 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3b6a16-d802-496d-bd19-b4f0a9c5f42b None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:32.341643 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3b6a16-d802-496d-bd19-b4f0a9c5f42b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:32.341985 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c695be85-d70e-4a55-83a7-4baf73e4e56a None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:32.355100 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3b6a16-d802-496d-bd19-b4f0a9c5f42b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:30:32.355316 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 37/71] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Jun 15 14:30:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:32.356244 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c695be85-d70e-4a55-83a7-4baf73e4e56a None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:32.363876 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-ab3b6a16-d802-496d-bd19-b4f0a9c5f42b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:30:32.363876 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:32.363876 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:32.363876 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:32.363876 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:32.363876 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:32.363876 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:32.363876 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:32.363876 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:32.363876 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:32.363876 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:32.363876 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:32.363876 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:30:32.363876 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:30:32.363876 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:30:32.363876 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:30:32.364774 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:30:32.364774 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:30:32.364774 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:30:32.364774 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:30:32.364774 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:32.364774 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:32.364774 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:32.364774 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:32.364774 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:32.364774 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:32.364774 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:32.364774 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:32.364774 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:32.364774 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:32.364774 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:32.364774 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:32.365532 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:32.365532 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:32.365532 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:32.365532 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:32.365532 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:30:32.365532 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:30:32.365532 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:30:32.365532 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:30:32.365532 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:30:32.365532 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:30:32.365532 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:30:32.365532 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:30:32.365532 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 35/72] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jun 15 14:30:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:32.366338 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058d9274-97cf-4a05-bcca-066714237f5f None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:32.366771 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058d9274-97cf-4a05-bcca-066714237f5f None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:32.367207 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058d9274-97cf-4a05-bcca-066714237f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:32.385003 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-058d9274-97cf-4a05-bcca-066714237f5f None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:32.386221 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-058d9274-97cf-4a05-bcca-066714237f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:32.440136 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:32.457256 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 36/73] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:30:32.458576 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f763be18-baf4-4ebe-9994-03b0edda77e4 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:32.458872 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f763be18-baf4-4ebe-9994-03b0edda77e4 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:32.459149 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f763be18-baf4-4ebe-9994-03b0edda77e4 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:32.460632 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 37/74] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:30:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:32.461506 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f763be18-baf4-4ebe-9994-03b0edda77e4 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:32.472127 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c695be85-d70e-4a55-83a7-4baf73e4e56a None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:32.512300 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35bf09a-1c12-4b0c-a374-4ddb7549253c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:32.513356 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35bf09a-1c12-4b0c-a374-4ddb7549253c None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:32.513643 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35bf09a-1c12-4b0c-a374-4ddb7549253c None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:32.514243 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35bf09a-1c12-4b0c-a374-4ddb7549253c None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:32.515743 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35bf09a-1c12-4b0c-a374-4ddb7549253c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:32.516068 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35bf09a-1c12-4b0c-a374-4ddb7549253c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:32.517477 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35bf09a-1c12-4b0c-a374-4ddb7549253c None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:32.526485 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 38/75] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jun 15 14:30:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:32.527310 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35bf09a-1c12-4b0c-a374-4ddb7549253c None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:32.554147 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:32.585009 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f763be18-baf4-4ebe-9994-03b0edda77e4 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:32.629136 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ec39ea-3fe4-4c4f-8add-b1b03aab2c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:32.630236 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec39ea-3fe4-4c4f-8add-b1b03aab2c13 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:32.630689 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec39ea-3fe4-4c4f-8add-b1b03aab2c13 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:32.630976 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec39ea-3fe4-4c4f-8add-b1b03aab2c13 None admin] PATH_INFO: `/v3/users/7fa722bf1210411e9101bfd161c43eaa` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:32.643358 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35bf09a-1c12-4b0c-a374-4ddb7549253c None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:32.645535 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ec39ea-3fe4-4c4f-8add-b1b03aab2c13 None admin] RBAC: Authorizing `identity:get_user(user_id=7fa722bf1210411e9101bfd161c43eaa)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:32.647288 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ec39ea-3fe4-4c4f-8add-b1b03aab2c13 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:32.649169 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 38/76] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jun 15 14:30:32 2022] GET /identity/v3/users/7fa722bf1210411e9101bfd161c43eaa => generated 275 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:32.651010 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff4d0ee-06f3-45e7-b904-8eec07e9a323 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:32.651387 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff4d0ee-06f3-45e7-b904-8eec07e9a323 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:32.651911 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff4d0ee-06f3-45e7-b904-8eec07e9a323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:32.670206 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-0ff4d0ee-06f3-45e7-b904-8eec07e9a323 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:32.672031 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-0ff4d0ee-06f3-45e7-b904-8eec07e9a323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:32.682310 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d814c140-9c94-466d-b25b-9522a20e8cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:32.682983 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d814c140-9c94-466d-b25b-9522a20e8cee None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:32.683273 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d814c140-9c94-466d-b25b-9522a20e8cee None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:32.683559 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d814c140-9c94-466d-b25b-9522a20e8cee None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:32.684383 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d814c140-9c94-466d-b25b-9522a20e8cee None admin] RBAC: Adding query filter params (name=admin) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:32.684785 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d814c140-9c94-466d-b25b-9522a20e8cee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:32.686166 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d814c140-9c94-466d-b25b-9522a20e8cee None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:32.694892 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 39/77] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jun 15 14:30:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:32.695946 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d814c140-9c94-466d-b25b-9522a20e8cee None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:32.712443 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:32.753137 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 39/78] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:30:32.753986 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff4d0ee-06f3-45e7-b904-8eec07e9a323 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:32.788721 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:32.813123 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d814c140-9c94-466d-b25b-9522a20e8cee None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:32.852768 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb7c6b2-17a4-4218-8c8e-9d4bbd227f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:32.853803 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb7c6b2-17a4-4218-8c8e-9d4bbd227f6d None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:32.854242 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb7c6b2-17a4-4218-8c8e-9d4bbd227f6d None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:32.854655 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb7c6b2-17a4-4218-8c8e-9d4bbd227f6d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:32.865358 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff4d0ee-06f3-45e7-b904-8eec07e9a323 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:32.867715 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb7c6b2-17a4-4218-8c8e-9d4bbd227f6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:30:32.876644 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-3cb7c6b2-17a4-4218-8c8e-9d4bbd227f6d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:30:32.876644 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:32.876644 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:32.876644 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:32.876644 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:32.876644 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:32.876644 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:32.876644 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:32.876644 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:32.876644 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:32.876644 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:32.876644 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:32.876644 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:30:32.876644 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:30:32.876644 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:30:32.876644 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:30:32.877563 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:30:32.877563 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:30:32.877563 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:30:32.877563 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:30:32.877563 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:32.877563 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:32.877563 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:32.877563 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:32.877563 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:32.877563 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:32.877563 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:32.877563 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:32.877563 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:32.877563 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:32.877563 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:32.877563 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:32.878310 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:32.878310 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:32.878310 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:32.878310 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:32.878310 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:30:32.878310 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:30:32.878310 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:30:32.878310 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:30:32.878310 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:30:32.878310 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:30:32.878310 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:30:32.878310 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:30:32.878310 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 40/79] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jun 15 14:30:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:32.878930 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb7c6b2-17a4-4218-8c8e-9d4bbd227f6d None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:32.903466 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3283af5a-856a-40c2-bf9f-16aabf3f39b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:32.904352 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3283af5a-856a-40c2-bf9f-16aabf3f39b2 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:32.904639 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3283af5a-856a-40c2-bf9f-16aabf3f39b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:32.904913 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3283af5a-856a-40c2-bf9f-16aabf3f39b2 None admin] PATH_INFO: `/v3/users/7fa722bf1210411e9101bfd161c43eaa` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:32.906211 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3283af5a-856a-40c2-bf9f-16aabf3f39b2 None admin] RBAC: Authorizing `identity:get_user(user_id=7fa722bf1210411e9101bfd161c43eaa)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:32.907666 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3283af5a-856a-40c2-bf9f-16aabf3f39b2 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:32.909403 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 40/80] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jun 15 14:30:32 2022] GET /identity/v3/users/7fa722bf1210411e9101bfd161c43eaa => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:32.909403 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3283af5a-856a-40c2-bf9f-16aabf3f39b2 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:32.954362 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:32.987296 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb7c6b2-17a4-4218-8c8e-9d4bbd227f6d None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:33.019294 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3283af5a-856a-40c2-bf9f-16aabf3f39b2 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:33.025676 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6de8a2f-2820-4c0d-bab6-4315239bd49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:33.026332 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6de8a2f-2820-4c0d-bab6-4315239bd49f None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:33.026623 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6de8a2f-2820-4c0d-bab6-4315239bd49f None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:33.026913 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6de8a2f-2820-4c0d-bab6-4315239bd49f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:33.040377 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:33.056260 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba92044-83f9-4630-b3da-08f3ddf18d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:33.057143 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba92044-83f9-4630-b3da-08f3ddf18d38 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:33.057467 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba92044-83f9-4630-b3da-08f3ddf18d38 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:33.057691 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba92044-83f9-4630-b3da-08f3ddf18d38 None admin] PATH_INFO: `/v3/users/11d932a20a0f4d8b9ce5ee15655d936c` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:33.058887 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba92044-83f9-4630-b3da-08f3ddf18d38 None admin] RBAC: Authorizing `identity:get_user(user_id=11d932a20a0f4d8b9ce5ee15655d936c)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:33.060387 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba92044-83f9-4630-b3da-08f3ddf18d38 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:33.063284 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 41/81] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jun 15 14:30:32 2022] GET /identity/v3/users/11d932a20a0f4d8b9ce5ee15655d936c => generated 247 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:33.065255 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba92044-83f9-4630-b3da-08f3ddf18d38 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:33.104804 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6de8a2f-2820-4c0d-bab6-4315239bd49f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:33.106407 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6de8a2f-2820-4c0d-bab6-4315239bd49f None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:33.176648 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba92044-83f9-4630-b3da-08f3ddf18d38 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:33.184458 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:33.211566 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac4d0d8-1b1c-4e7f-8e2a-9ccf5984f3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:33.212645 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac4d0d8-1b1c-4e7f-8e2a-9ccf5984f3d9 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:33.212933 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac4d0d8-1b1c-4e7f-8e2a-9ccf5984f3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:33.213216 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac4d0d8-1b1c-4e7f-8e2a-9ccf5984f3d9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:33.214628 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac4d0d8-1b1c-4e7f-8e2a-9ccf5984f3d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:33.216274 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac4d0d8-1b1c-4e7f-8e2a-9ccf5984f3d9 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:33.216975 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 41/82] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jun 15 14:30:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:30:33.218951 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 42/83] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:33.219065 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6de8a2f-2820-4c0d-bab6-4315239bd49f None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:33.220934 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac4d0d8-1b1c-4e7f-8e2a-9ccf5984f3d9 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:33.269918 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:33.333666 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6de8a2f-2820-4c0d-bab6-4315239bd49f None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:33.334500 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac4d0d8-1b1c-4e7f-8e2a-9ccf5984f3d9 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:33.372709 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226b3541-0403-49e8-a00f-e0d22652ec2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:33.373388 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b3541-0403-49e8-a00f-e0d22652ec2a None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:33.373548 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b0a325-432e-48b4-88d3-e82c097c0f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:33.373666 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b3541-0403-49e8-a00f-e0d22652ec2a None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:33.374495 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b3541-0403-49e8-a00f-e0d22652ec2a None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:33.375226 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b0a325-432e-48b4-88d3-e82c097c0f4c None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:33.375524 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b0a325-432e-48b4-88d3-e82c097c0f4c None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:33.375754 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226b3541-0403-49e8-a00f-e0d22652ec2a None admin] RBAC: Adding query filter params (name=member) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:33.375848 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b0a325-432e-48b4-88d3-e82c097c0f4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:33.376075 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226b3541-0403-49e8-a00f-e0d22652ec2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:33.377608 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226b3541-0403-49e8-a00f-e0d22652ec2a None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:33.386387 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 42/84] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Jun 15 14:30:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:33.387022 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226b3541-0403-49e8-a00f-e0d22652ec2a None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:33.428680 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:33.456164 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b0a325-432e-48b4-88d3-e82c097c0f4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:33.457620 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b0a325-432e-48b4-88d3-e82c097c0f4c None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:33.506260 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226b3541-0403-49e8-a00f-e0d22652ec2a None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:33.508620 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:33.554044 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f018792-899b-490d-bfd2-9cec81a7b72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:33.554380 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f018792-899b-490d-bfd2-9cec81a7b72a None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:33.554660 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f018792-899b-490d-bfd2-9cec81a7b72a None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:33.556684 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f018792-899b-490d-bfd2-9cec81a7b72a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:33.571844 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 43/85] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jun 15 14:30:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:30:33.574627 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b0a325-432e-48b4-88d3-e82c097c0f4c None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:33.635557 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f018792-899b-490d-bfd2-9cec81a7b72a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:33.637076 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f018792-899b-490d-bfd2-9cec81a7b72a None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:33.655091 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:33.683406 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b0a325-432e-48b4-88d3-e82c097c0f4c None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:33.720501 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46326fa-0eae-4fcd-bd27-6f2034e0a8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:33.721383 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46326fa-0eae-4fcd-bd27-6f2034e0a8bd None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:33.721677 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46326fa-0eae-4fcd-bd27-6f2034e0a8bd None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:33.721966 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46326fa-0eae-4fcd-bd27-6f2034e0a8bd None admin] PATH_INFO: `/v3/projects/af60304e389b4295adfafbfc85459962` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:33.724836 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46326fa-0eae-4fcd-bd27-6f2034e0a8bd None admin] RBAC: Authorizing `identity:get_project(project_id=af60304e389b4295adfafbfc85459962)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:33.726310 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46326fa-0eae-4fcd-bd27-6f2034e0a8bd None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:33.728050 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 44/86] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jun 15 14:30:33 2022] GET /identity/v3/projects/af60304e389b4295adfafbfc85459962 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:33.734399 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46326fa-0eae-4fcd-bd27-6f2034e0a8bd None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:33.750874 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 43/87] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jun 15 14:30:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:30:33.751732 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f018792-899b-490d-bfd2-9cec81a7b72a None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:33.755327 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:33.847682 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46326fa-0eae-4fcd-bd27-6f2034e0a8bd None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:33.863712 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f018792-899b-490d-bfd2-9cec81a7b72a None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:33.887225 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a49b13e-c8bb-4c5f-8415-d3b5e012602e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:33.888160 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49b13e-c8bb-4c5f-8415-d3b5e012602e None admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:33.888448 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49b13e-c8bb-4c5f-8415-d3b5e012602e None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:33.888722 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49b13e-c8bb-4c5f-8415-d3b5e012602e None admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:33.889734 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a49b13e-c8bb-4c5f-8415-d3b5e012602e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:33.891561 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a49b13e-c8bb-4c5f-8415-d3b5e012602e None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:33.900269 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181a0cab-4d0c-42f5-ade5-a449228b1c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:33.901482 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181a0cab-4d0c-42f5-ade5-a449228b1c30 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:33.901769 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181a0cab-4d0c-42f5-ade5-a449228b1c30 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:33.902042 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181a0cab-4d0c-42f5-ade5-a449228b1c30 None admin] PATH_INFO: `/v3/users/7fa722bf1210411e9101bfd161c43eaa` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:33.903766 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181a0cab-4d0c-42f5-ade5-a449228b1c30 None admin] RBAC: Authorizing `identity:get_user(user_id=7fa722bf1210411e9101bfd161c43eaa)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:33.905842 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181a0cab-4d0c-42f5-ade5-a449228b1c30 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:33.907469 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 44/88] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jun 15 14:30:33 2022] GET /identity/v3/users/7fa722bf1210411e9101bfd161c43eaa => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:33.908246 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181a0cab-4d0c-42f5-ade5-a449228b1c30 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:33.949222 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:33.955201 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 45/89] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Jun 15 14:30:33 2022] POST /identity/v3/users => generated 289 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:30:33.956073 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a49b13e-c8bb-4c5f-8415-d3b5e012602e None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:33.991638 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:34.035371 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181a0cab-4d0c-42f5-ade5-a449228b1c30 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:34.085530 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34368578-d1d2-48be-a0af-c42858db88a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:34.086802 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34368578-d1d2-48be-a0af-c42858db88a0 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:34.087445 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34368578-d1d2-48be-a0af-c42858db88a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:34.087927 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34368578-d1d2-48be-a0af-c42858db88a0 None admin] PATH_INFO: `/v3/projects/af60304e389b4295adfafbfc85459962` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:34.089526 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34368578-d1d2-48be-a0af-c42858db88a0 None admin] RBAC: Authorizing `identity:get_project(project_id=af60304e389b4295adfafbfc85459962)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:34.089738 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a49b13e-c8bb-4c5f-8415-d3b5e012602e None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:34.091472 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34368578-d1d2-48be-a0af-c42858db88a0 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:34.094036 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 45/90] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jun 15 14:30:33 2022] GET /identity/v3/projects/af60304e389b4295adfafbfc85459962 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:34.095106 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34368578-d1d2-48be-a0af-c42858db88a0 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:34.145322 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c148c9-20bc-4ffd-a35c-e8035449655c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:34.145322 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c148c9-20bc-4ffd-a35c-e8035449655c None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:34.145322 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c148c9-20bc-4ffd-a35c-e8035449655c None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:34.145650 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c148c9-20bc-4ffd-a35c-e8035449655c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:34.146343 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c148c9-20bc-4ffd-a35c-e8035449655c None admin] RBAC: Adding query filter params (role.id=9effaf496b874d01bd3f6d0555ef2d69, scope.project.id=af60304e389b4295adfafbfc85459962, user.id=7fa722bf1210411e9101bfd161c43eaa) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:34.146675 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c148c9-20bc-4ffd-a35c-e8035449655c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:34.149575 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c148c9-20bc-4ffd-a35c-e8035449655c None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:34.159070 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 46/91] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jun 15 14:30:33 2022] GET /identity/v3/role_assignments?user.id=7fa722bf1210411e9101bfd161c43eaa&scope.project.id=af60304e389b4295adfafbfc85459962&role.id=9effaf496b874d01bd3f6d0555ef2d69 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:34.160507 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c148c9-20bc-4ffd-a35c-e8035449655c None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:34.253023 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34368578-d1d2-48be-a0af-c42858db88a0 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:34.294384 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:34.301888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fe4d8d-7b1d-46a2-bbc0-50df7a00a4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:34.303017 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fe4d8d-7b1d-46a2-bbc0-50df7a00a4ac None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:34.303445 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fe4d8d-7b1d-46a2-bbc0-50df7a00a4ac None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:34.303867 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fe4d8d-7b1d-46a2-bbc0-50df7a00a4ac None admin] PATH_INFO: `/v3/projects/af60304e389b4295adfafbfc85459962` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:34.307768 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fe4d8d-7b1d-46a2-bbc0-50df7a00a4ac None admin] RBAC: Authorizing `identity:get_project(project_id=af60304e389b4295adfafbfc85459962)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:34.309505 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fe4d8d-7b1d-46a2-bbc0-50df7a00a4ac None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:34.321670 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 46/92] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jun 15 14:30:34 2022] GET /identity/v3/projects/af60304e389b4295adfafbfc85459962 => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:34.321670 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fe4d8d-7b1d-46a2-bbc0-50df7a00a4ac None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:34.332669 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:34.351017 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c148c9-20bc-4ffd-a35c-e8035449655c None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:34.406993 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f30e6cf-893a-461c-b2cc-ce5f45176d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:34.410839 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f30e6cf-893a-461c-b2cc-ce5f45176d0b None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:34.410839 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f30e6cf-893a-461c-b2cc-ce5f45176d0b None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:34.410839 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f30e6cf-893a-461c-b2cc-ce5f45176d0b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:34.481377 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fe4d8d-7b1d-46a2-bbc0-50df7a00a4ac None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:34.513916 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f30e6cf-893a-461c-b2cc-ce5f45176d0b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:34.516671 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f30e6cf-893a-461c-b2cc-ce5f45176d0b None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:34.519298 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdb3dc6-0965-421d-becf-d61132b2ccc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:34.520068 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdb3dc6-0965-421d-becf-d61132b2ccc3 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:34.520539 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdb3dc6-0965-421d-becf-d61132b2ccc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:34.520945 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdb3dc6-0965-421d-becf-d61132b2ccc3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:34.521955 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdb3dc6-0965-421d-becf-d61132b2ccc3 None admin] RBAC: Adding query filter params (role.id=61e98f694ee741a9894b7e6db6dd11d3, scope.project.id=af60304e389b4295adfafbfc85459962, user.id=7fa722bf1210411e9101bfd161c43eaa) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:34.522466 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdb3dc6-0965-421d-becf-d61132b2ccc3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:34.524473 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdb3dc6-0965-421d-becf-d61132b2ccc3 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:34.535719 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 47/93] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jun 15 14:30:34 2022] GET /identity/v3/role_assignments?user.id=7fa722bf1210411e9101bfd161c43eaa&scope.project.id=af60304e389b4295adfafbfc85459962&role.id=61e98f694ee741a9894b7e6db6dd11d3 => generated 259 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:34.536865 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdb3dc6-0965-421d-becf-d61132b2ccc3 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:34.647412 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:34.652605 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:34.716347 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdb3dc6-0965-421d-becf-d61132b2ccc3 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:34.747143 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 47/94] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jun 15 14:30:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 587 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:30:34.753458 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f30e6cf-893a-461c-b2cc-ce5f45176d0b None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:34.800174 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2e37c5-ccc1-4cc8-845d-29507396b396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:34.802804 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e37c5-ccc1-4cc8-845d-29507396b396 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:34.804031 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e37c5-ccc1-4cc8-845d-29507396b396 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:34.804516 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e37c5-ccc1-4cc8-845d-29507396b396 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:34.806250 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2e37c5-ccc1-4cc8-845d-29507396b396 None admin] RBAC: Adding query filter params (role.id=2bd1979f53bb4853a50dd5bee6aa0782, scope.project.id=af60304e389b4295adfafbfc85459962, user.id=11d932a20a0f4d8b9ce5ee15655d936c) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:34.806942 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2e37c5-ccc1-4cc8-845d-29507396b396 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:34.809858 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2e37c5-ccc1-4cc8-845d-29507396b396 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:34.819879 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 48/95] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jun 15 14:30:34 2022] GET /identity/v3/role_assignments?user.id=11d932a20a0f4d8b9ce5ee15655d936c&scope.project.id=af60304e389b4295adfafbfc85459962&role.id=2bd1979f53bb4853a50dd5bee6aa0782 => generated 259 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:34.907809 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f30e6cf-893a-461c-b2cc-ce5f45176d0b None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:34.948689 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798665ec-2a80-4b13-9ef2-2b7d4db97f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:34.949889 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798665ec-2a80-4b13-9ef2-2b7d4db97f74 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:34.950350 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798665ec-2a80-4b13-9ef2-2b7d4db97f74 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:34.950862 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798665ec-2a80-4b13-9ef2-2b7d4db97f74 None admin] PATH_INFO: `/v3/projects/4cf55663d7eb45fd8ef9be9ec086db23` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:34.953011 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798665ec-2a80-4b13-9ef2-2b7d4db97f74 None admin] RBAC: Authorizing `identity:get_project(project_id=4cf55663d7eb45fd8ef9be9ec086db23)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:34.954878 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798665ec-2a80-4b13-9ef2-2b7d4db97f74 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:34.957457 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 48/96] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jun 15 14:30:34 2022] GET /identity/v3/projects/4cf55663d7eb45fd8ef9be9ec086db23 => generated 312 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:34.965465 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2e37c5-ccc1-4cc8-845d-29507396b396 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:35.099037 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2e37c5-ccc1-4cc8-845d-29507396b396 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:35.159234 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f6a2e4-097a-46da-89a4-8d8bafc16927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:35.160425 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f6a2e4-097a-46da-89a4-8d8bafc16927 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:35.161021 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f6a2e4-097a-46da-89a4-8d8bafc16927 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:35.161533 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f6a2e4-097a-46da-89a4-8d8bafc16927 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:35.162634 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f6a2e4-097a-46da-89a4-8d8bafc16927 None admin] RBAC: Adding query filter params (role.id=9effaf496b874d01bd3f6d0555ef2d69, scope.project.id=4cf55663d7eb45fd8ef9be9ec086db23, user.id=7fa722bf1210411e9101bfd161c43eaa) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:35.163144 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f6a2e4-097a-46da-89a4-8d8bafc16927 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:35.169025 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f6a2e4-097a-46da-89a4-8d8bafc16927 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:35.177214 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:35.180332 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 49/97] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jun 15 14:30:34 2022] GET /identity/v3/role_assignments?user.id=7fa722bf1210411e9101bfd161c43eaa&scope.project.id=4cf55663d7eb45fd8ef9be9ec086db23&role.id=9effaf496b874d01bd3f6d0555ef2d69 => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:37.610559 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590cee68-dd09-44cb-a6f2-aef856fda63d None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:37.611099 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590cee68-dd09-44cb-a6f2-aef856fda63d None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:37.611099 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590cee68-dd09-44cb-a6f2-aef856fda63d None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:37.612701 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 49/98] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:30:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:37.618302 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b375c7-21f3-4a7a-babb-d742d2b043f3 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:37.618752 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b375c7-21f3-4a7a-babb-d742d2b043f3 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:37.618948 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b375c7-21f3-4a7a-babb-d742d2b043f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:37.641405 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-e9b375c7-21f3-4a7a-babb-d742d2b043f3 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:37.642641 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e9b375c7-21f3-4a7a-babb-d742d2b043f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:37.742165 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 50/99] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:30:37.787782 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fcf614-6db5-4e29-a466-dfdcaf01fdd3 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:37.787782 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fcf614-6db5-4e29-a466-dfdcaf01fdd3 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:37.787782 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fcf614-6db5-4e29-a466-dfdcaf01fdd3 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:37.787782 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 50/100] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:30:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:37.789295 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b375c7-21f3-4a7a-babb-d742d2b043f3 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:37.806546 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb394f9-77c0-44a4-800c-18a6d63bc3ca None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:37.807585 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb394f9-77c0-44a4-800c-18a6d63bc3ca None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:37.808098 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb394f9-77c0-44a4-800c-18a6d63bc3ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:37.829486 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-3cb394f9-77c0-44a4-800c-18a6d63bc3ca None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:37.833089 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-3cb394f9-77c0-44a4-800c-18a6d63bc3ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:37.868266 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:37.931372 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 51/101] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:30:37.933253 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a169fbfa-d786-4d51-ad60-c38003ed2fec None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:37.933899 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a169fbfa-d786-4d51-ad60-c38003ed2fec None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:37.935680 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a169fbfa-d786-4d51-ad60-c38003ed2fec None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:37.937872 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 52/102] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:30:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:37.946164 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb81a3f6-851b-443a-b7f0-b190d104ad09 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:37.946841 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb81a3f6-851b-443a-b7f0-b190d104ad09 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:37.947341 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb81a3f6-851b-443a-b7f0-b190d104ad09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:37.969606 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-bb81a3f6-851b-443a-b7f0-b190d104ad09 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:37.973046 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-bb81a3f6-851b-443a-b7f0-b190d104ad09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:38.032138 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b375c7-21f3-4a7a-babb-d742d2b043f3 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:38.060574 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 53/103] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:30:38.062662 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb81a3f6-851b-443a-b7f0-b190d104ad09 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:38.111347 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:38.136991 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a9aa56-d130-4236-98e7-f289e90cabad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:38.138096 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a9aa56-d130-4236-98e7-f289e90cabad None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:38.138670 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a9aa56-d130-4236-98e7-f289e90cabad None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:38.139085 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a9aa56-d130-4236-98e7-f289e90cabad None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:38.172742 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a9aa56-d130-4236-98e7-f289e90cabad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:30:38.189176 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-40a9aa56-d130-4236-98e7-f289e90cabad None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:30:38.189176 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:38.189176 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:38.189176 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:38.189176 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:38.189176 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:38.189176 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:38.189176 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:38.189176 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:38.189176 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:38.189176 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:38.189176 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:38.189176 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:30:38.189176 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:30:38.189176 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:30:38.189176 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:30:38.190113 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:30:38.190113 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:30:38.190113 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:30:38.190113 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:30:38.190113 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:38.190113 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:38.190113 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:38.190113 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:38.190113 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:38.190113 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:38.190113 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:38.190113 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:38.190113 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:38.190113 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:38.190113 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:38.190113 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:38.190951 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:38.190951 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:38.190951 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:38.190951 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:38.190951 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:30:38.190951 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:30:38.190951 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:30:38.190951 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:30:38.190951 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:30:38.190951 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:30:38.190951 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:30:38.190951 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:30:38.198714 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 51/104] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jun 15 14:30:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 405 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:38.199873 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a9aa56-d130-4236-98e7-f289e90cabad None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:38.204905 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb81a3f6-851b-443a-b7f0-b190d104ad09 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:38.226676 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:38.261304 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9c7c15-280c-48b3-b37a-2be3aa99f0f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:38.262111 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9c7c15-280c-48b3-b37a-2be3aa99f0f1 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:38.262398 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9c7c15-280c-48b3-b37a-2be3aa99f0f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:38.262676 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9c7c15-280c-48b3-b37a-2be3aa99f0f1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:38.266155 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9c7c15-280c-48b3-b37a-2be3aa99f0f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:38.270020 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9c7c15-280c-48b3-b37a-2be3aa99f0f1 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:38.272293 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 54/105] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:38.272899 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9c7c15-280c-48b3-b37a-2be3aa99f0f1 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:38.392994 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9c7c15-280c-48b3-b37a-2be3aa99f0f1 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:38.442548 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21775d39-2301-49cc-987d-c7efee3fe5fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:38.443256 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21775d39-2301-49cc-987d-c7efee3fe5fe None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:38.443547 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21775d39-2301-49cc-987d-c7efee3fe5fe None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:38.443835 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21775d39-2301-49cc-987d-c7efee3fe5fe None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:38.446256 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21775d39-2301-49cc-987d-c7efee3fe5fe None admin] RBAC: Adding query filter params (name=reader) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:38.446582 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21775d39-2301-49cc-987d-c7efee3fe5fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:38.450793 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21775d39-2301-49cc-987d-c7efee3fe5fe None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:38.462576 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 55/106] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Jun 15 14:30:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:38.463226 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21775d39-2301-49cc-987d-c7efee3fe5fe None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:38.490307 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:38.549533 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a9aa56-d130-4236-98e7-f289e90cabad None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:38.595167 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43513303-dd74-4479-94ec-60ba6f7391fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:38.595957 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43513303-dd74-4479-94ec-60ba6f7391fa None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:38.596290 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43513303-dd74-4479-94ec-60ba6f7391fa None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:38.596567 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43513303-dd74-4479-94ec-60ba6f7391fa None admin] PATH_INFO: `/v3/roles/member` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:38.607804 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21775d39-2301-49cc-987d-c7efee3fe5fe None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:38.608365 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43513303-dd74-4479-94ec-60ba6f7391fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:30:38.615989 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-43513303-dd74-4479-94ec-60ba6f7391fa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:30:38.615989 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:38.615989 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:38.615989 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:38.615989 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:38.615989 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:38.615989 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:38.615989 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:38.615989 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:38.615989 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:38.615989 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:38.615989 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:38.615989 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:30:38.615989 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:30:38.615989 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:30:38.615989 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:30:38.616887 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:30:38.616887 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:30:38.616887 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:30:38.616887 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:30:38.616887 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:38.616887 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:38.616887 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:38.616887 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:38.616887 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:38.616887 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:38.616887 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:38.616887 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:38.616887 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:38.616887 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:38.616887 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:38.616887 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:38.617652 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:38.617652 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:38.617652 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:38.617652 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:38.617652 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:30:38.617652 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:30:38.617652 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:30:38.617652 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:30:38.617652 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:30:38.617652 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:30:38.617652 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:30:38.617652 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:30:38.617652 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 52/107] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jun 15 14:30:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 418 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:38.621430 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43513303-dd74-4479-94ec-60ba6f7391fa None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:38.651290 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8645fb5a-79ba-4144-80ff-45603e8e3bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:38.652629 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8645fb5a-79ba-4144-80ff-45603e8e3bbb None admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:38.653175 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8645fb5a-79ba-4144-80ff-45603e8e3bbb None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:38.653355 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8645fb5a-79ba-4144-80ff-45603e8e3bbb None admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:38.654062 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8645fb5a-79ba-4144-80ff-45603e8e3bbb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:38.655810 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8645fb5a-79ba-4144-80ff-45603e8e3bbb None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:38.677216 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:38.728149 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43513303-dd74-4479-94ec-60ba6f7391fa None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:38.736189 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 56/108] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Jun 15 14:30:38 2022] POST /identity/v3/projects => generated 302 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:30:38.736189 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5127640-b91a-430e-8d9d-89b478bec530 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:38.736189 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5127640-b91a-430e-8d9d-89b478bec530 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:38.736189 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5127640-b91a-430e-8d9d-89b478bec530 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:38.740231 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 57/109] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:30:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:38.740231 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52659fda-166d-4d7e-819f-3f04253ff072 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:38.740231 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52659fda-166d-4d7e-819f-3f04253ff072 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:38.740446 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52659fda-166d-4d7e-819f-3f04253ff072 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:38.744164 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 58/110] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:30:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:38.744164 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bf85a7-2dcb-4eba-b79b-a5504c8673a5 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:38.744164 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bf85a7-2dcb-4eba-b79b-a5504c8673a5 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:38.744164 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bf85a7-2dcb-4eba-b79b-a5504c8673a5 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:38.745352 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 59/111] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:30:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:38.745747 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bf85a7-2dcb-4eba-b79b-a5504c8673a5 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:38.783061 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3142a19c-93e5-4d3d-a1e0-a257b18b7be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:38.784652 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3142a19c-93e5-4d3d-a1e0-a257b18b7be5 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:38.785152 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3142a19c-93e5-4d3d-a1e0-a257b18b7be5 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:38.785632 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3142a19c-93e5-4d3d-a1e0-a257b18b7be5 None admin] PATH_INFO: `/v3/users/aaa9a41e3d594efaaa8a9bd29ce1644e` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:38.805649 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3142a19c-93e5-4d3d-a1e0-a257b18b7be5 None admin] RBAC: Authorizing `identity:get_user(user_id=aaa9a41e3d594efaaa8a9bd29ce1644e)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:38.807589 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3142a19c-93e5-4d3d-a1e0-a257b18b7be5 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:38.810026 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 53/112] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jun 15 14:30:38 2022] GET /identity/v3/users/aaa9a41e3d594efaaa8a9bd29ce1644e => generated 289 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:38.817289 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52f2f01-9d92-44f1-97ee-97982dff948d None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:38.817898 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52f2f01-9d92-44f1-97ee-97982dff948d None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:38.818431 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52f2f01-9d92-44f1-97ee-97982dff948d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:38.835614 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:38.838768 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-b52f2f01-9d92-44f1-97ee-97982dff948d None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:38.840352 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-b52f2f01-9d92-44f1-97ee-97982dff948d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:38.877070 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bf85a7-2dcb-4eba-b79b-a5504c8673a5 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:38.935404 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7983d231-6784-43fa-bd7b-efe7fd72d2a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:38.935404 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7983d231-6784-43fa-bd7b-efe7fd72d2a3 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:38.935404 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7983d231-6784-43fa-bd7b-efe7fd72d2a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:38.935404 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7983d231-6784-43fa-bd7b-efe7fd72d2a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:38.935785 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7983d231-6784-43fa-bd7b-efe7fd72d2a3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:38.935785 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7983d231-6784-43fa-bd7b-efe7fd72d2a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:38.935785 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7983d231-6784-43fa-bd7b-efe7fd72d2a3 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:38.935965 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 54/113] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:30:38.935965 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1577e5b0-3537-4754-af78-3a3f395931c9 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:38.935965 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1577e5b0-3537-4754-af78-3a3f395931c9 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:38.935965 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1577e5b0-3537-4754-af78-3a3f395931c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:38.942216 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 60/114] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Jun 15 14:30:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:38.943488 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10fcfbf-20fa-4b0f-b555-f0aa38242540 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:38.943783 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10fcfbf-20fa-4b0f-b555-f0aa38242540 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:38.944049 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10fcfbf-20fa-4b0f-b555-f0aa38242540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:38.960767 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-1577e5b0-3537-4754-af78-3a3f395931c9 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:38.961884 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-1577e5b0-3537-4754-af78-3a3f395931c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:38.970239 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-e10fcfbf-20fa-4b0f-b555-f0aa38242540 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:38.972032 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-e10fcfbf-20fa-4b0f-b555-f0aa38242540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:39.002247 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:39.052265 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 55/115] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:30:39.053035 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1577e5b0-3537-4754-af78-3a3f395931c9 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:39.073900 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 61/116] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:30:39.074778 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10fcfbf-20fa-4b0f-b555-f0aa38242540 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:39.175861 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:39.207741 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10fcfbf-20fa-4b0f-b555-f0aa38242540 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:39.228945 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1577e5b0-3537-4754-af78-3a3f395931c9 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:39.248164 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab185715-45cc-4f42-9ed0-6a8bcae981c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:39.249092 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab185715-45cc-4f42-9ed0-6a8bcae981c5 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:39.249379 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab185715-45cc-4f42-9ed0-6a8bcae981c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:39.249660 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab185715-45cc-4f42-9ed0-6a8bcae981c5 None admin] PATH_INFO: `/v3/users/7fa722bf1210411e9101bfd161c43eaa` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:39.250994 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab185715-45cc-4f42-9ed0-6a8bcae981c5 None admin] RBAC: Authorizing `identity:get_user(user_id=7fa722bf1210411e9101bfd161c43eaa)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:39.252459 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab185715-45cc-4f42-9ed0-6a8bcae981c5 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:39.256151 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 62/117] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jun 15 14:30:39 2022] GET /identity/v3/users/7fa722bf1210411e9101bfd161c43eaa => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:39.256151 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab185715-45cc-4f42-9ed0-6a8bcae981c5 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:39.290793 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665a3ee6-28f3-41fe-b7d3-71238f8cd8e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:39.292238 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665a3ee6-28f3-41fe-b7d3-71238f8cd8e8 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:39.292903 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665a3ee6-28f3-41fe-b7d3-71238f8cd8e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:39.293348 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665a3ee6-28f3-41fe-b7d3-71238f8cd8e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:39.352317 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:39.388578 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665a3ee6-28f3-41fe-b7d3-71238f8cd8e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:39.390193 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665a3ee6-28f3-41fe-b7d3-71238f8cd8e8 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:39.423076 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab185715-45cc-4f42-9ed0-6a8bcae981c5 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:39.470552 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:39.477335 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94643474-30f6-4e86-91d7-ddf309495c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:39.483856 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94643474-30f6-4e86-91d7-ddf309495c69 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:39.483856 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94643474-30f6-4e86-91d7-ddf309495c69 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:39.483856 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94643474-30f6-4e86-91d7-ddf309495c69 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:39.494403 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94643474-30f6-4e86-91d7-ddf309495c69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:30:39.504178 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-94643474-30f6-4e86-91d7-ddf309495c69 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:30:39.504178 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:39.504178 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:39.504178 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:39.504178 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:39.504178 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:39.504178 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:39.504178 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:39.504178 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:39.504178 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:39.504178 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:39.504178 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:39.504178 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:30:39.504178 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:30:39.504178 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:30:39.504178 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:30:39.505098 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:30:39.505098 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:30:39.505098 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:30:39.505098 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:30:39.505098 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:39.505098 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:39.505098 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:39.505098 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:39.505098 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:39.505098 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:39.505098 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:39.505098 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:39.505098 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:39.505098 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:39.505098 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:39.505098 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:39.505830 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:39.505830 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:39.505830 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:39.505830 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:39.505830 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:30:39.505830 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:30:39.505830 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:30:39.505830 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:30:39.505830 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:30:39.505830 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:30:39.505830 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:30:39.505830 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:30:39.512237 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 63/118] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Jun 15 14:30:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:39.513295 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94643474-30f6-4e86-91d7-ddf309495c69 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:39.543986 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 56/119] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jun 15 14:30:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:30:39.545109 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665a3ee6-28f3-41fe-b7d3-71238f8cd8e8 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:39.649205 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:39.669896 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94643474-30f6-4e86-91d7-ddf309495c69 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:39.695192 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665a3ee6-28f3-41fe-b7d3-71238f8cd8e8 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:39.720804 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e95f62-23c0-43fa-93a8-e769cf7f5c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:39.721757 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e95f62-23c0-43fa-93a8-e769cf7f5c19 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:39.722240 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e95f62-23c0-43fa-93a8-e769cf7f5c19 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:39.722845 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e95f62-23c0-43fa-93a8-e769cf7f5c19 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:39.732638 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea5680b-5a7c-412c-84ec-e11d69810787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:39.733568 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea5680b-5a7c-412c-84ec-e11d69810787 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:39.734050 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea5680b-5a7c-412c-84ec-e11d69810787 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:39.734402 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea5680b-5a7c-412c-84ec-e11d69810787 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:39.737136 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e95f62-23c0-43fa-93a8-e769cf7f5c19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:30:39.745081 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea5680b-5a7c-412c-84ec-e11d69810787 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:30:39.746399 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-64e95f62-23c0-43fa-93a8-e769cf7f5c19 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:30:39.746399 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:39.746399 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:39.746399 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:39.746399 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:39.746399 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:39.746399 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:39.746399 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:39.746399 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:39.746399 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:39.746399 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:39.746399 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:39.746399 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:30:39.746399 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:30:39.746399 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:30:39.746399 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:30:39.747428 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:30:39.747428 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:30:39.747428 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:30:39.747428 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:30:39.747428 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:39.747428 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:39.747428 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:39.747428 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:39.747428 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:39.747428 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:39.747428 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:39.747428 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:39.747428 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:39.747428 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:39.747428 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:39.747428 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:39.748406 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:39.748406 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:39.748406 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:39.748406 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:39.748406 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:30:39.748406 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:30:39.748406 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:30:39.748406 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:30:39.748406 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:30:39.748406 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:30:39.748406 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:30:39.748406 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:30:39.749348 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 64/120] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jun 15 14:30:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:39.750244 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e95f62-23c0-43fa-93a8-e769cf7f5c19 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:39.759542 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-eea5680b-5a7c-412c-84ec-e11d69810787 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:30:39.759542 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:39.759542 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:39.759542 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:39.759542 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:39.759542 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:39.759542 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:39.759542 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:39.759542 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:39.759542 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:39.759542 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:39.759542 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:39.759542 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:30:39.759542 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:30:39.759542 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:30:39.759542 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:30:39.760698 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:30:39.760698 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:30:39.760698 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:30:39.760698 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:30:39.760698 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:39.760698 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:39.760698 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:39.760698 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:39.760698 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:39.760698 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:39.760698 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:39.760698 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:39.760698 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:39.760698 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:39.760698 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:39.760698 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:39.761522 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:39.761522 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:39.761522 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:39.761522 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:39.761522 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:30:39.761522 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:30:39.761522 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:30:39.761522 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:30:39.761522 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:30:39.761522 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:30:39.761522 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:30:39.761522 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:30:39.768748 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 57/121] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jun 15 14:30:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:39.769731 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea5680b-5a7c-412c-84ec-e11d69810787 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:39.817461 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:39.873275 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e95f62-23c0-43fa-93a8-e769cf7f5c19 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:39.904268 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea5680b-5a7c-412c-84ec-e11d69810787 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:39.915213 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219d0e54-f25a-448f-8c2d-77c94872d4c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:39.916541 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d0e54-f25a-448f-8c2d-77c94872d4c0 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:39.916934 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d0e54-f25a-448f-8c2d-77c94872d4c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:39.917318 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d0e54-f25a-448f-8c2d-77c94872d4c0 None admin] PATH_INFO: `/v3/projects/af60304e389b4295adfafbfc85459962` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:39.919277 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219d0e54-f25a-448f-8c2d-77c94872d4c0 None admin] RBAC: Authorizing `identity:get_project(project_id=af60304e389b4295adfafbfc85459962)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:39.922220 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219d0e54-f25a-448f-8c2d-77c94872d4c0 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:39.925284 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 65/122] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jun 15 14:30:39 2022] GET /identity/v3/projects/af60304e389b4295adfafbfc85459962 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:39.926320 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219d0e54-f25a-448f-8c2d-77c94872d4c0 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:39.939022 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:39.948969 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92462f23-6378-4df7-9b6b-b75c2efed586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:39.949647 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92462f23-6378-4df7-9b6b-b75c2efed586 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:39.955775 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92462f23-6378-4df7-9b6b-b75c2efed586 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:39.955775 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92462f23-6378-4df7-9b6b-b75c2efed586 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:39.955775 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92462f23-6378-4df7-9b6b-b75c2efed586 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:39.955775 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92462f23-6378-4df7-9b6b-b75c2efed586 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:39.955775 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92462f23-6378-4df7-9b6b-b75c2efed586 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:39.964707 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 58/123] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jun 15 14:30:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:39.966397 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92462f23-6378-4df7-9b6b-b75c2efed586 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:40.073366 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219d0e54-f25a-448f-8c2d-77c94872d4c0 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:40.097211 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:40.098044 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92462f23-6378-4df7-9b6b-b75c2efed586 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:40.115043 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cf7f6c-75b7-4a3e-81ca-90ddb66c187e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:40.116041 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf7f6c-75b7-4a3e-81ca-90ddb66c187e None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:40.116612 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf7f6c-75b7-4a3e-81ca-90ddb66c187e None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:40.116995 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf7f6c-75b7-4a3e-81ca-90ddb66c187e None admin] PATH_INFO: `/v3/projects/af60304e389b4295adfafbfc85459962` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:40.118549 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cf7f6c-75b7-4a3e-81ca-90ddb66c187e None admin] RBAC: Authorizing `identity:get_project(project_id=af60304e389b4295adfafbfc85459962)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:40.120208 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cf7f6c-75b7-4a3e-81ca-90ddb66c187e None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:40.122721 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 66/124] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jun 15 14:30:39 2022] GET /identity/v3/projects/af60304e389b4295adfafbfc85459962 => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:40.124367 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cf7f6c-75b7-4a3e-81ca-90ddb66c187e None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:40.136418 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee980c8b-c545-453b-bc71-02f9009cd873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:40.137545 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee980c8b-c545-453b-bc71-02f9009cd873 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:40.137838 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee980c8b-c545-453b-bc71-02f9009cd873 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:40.138120 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee980c8b-c545-453b-bc71-02f9009cd873 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:40.138878 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee980c8b-c545-453b-bc71-02f9009cd873 None admin] RBAC: Adding query filter params (name=member) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:40.139248 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee980c8b-c545-453b-bc71-02f9009cd873 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:40.141435 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee980c8b-c545-453b-bc71-02f9009cd873 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:40.149886 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 59/125] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Jun 15 14:30:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:40.150907 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee980c8b-c545-453b-bc71-02f9009cd873 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:40.213142 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:40.271224 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee980c8b-c545-453b-bc71-02f9009cd873 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:40.273432 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cf7f6c-75b7-4a3e-81ca-90ddb66c187e None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:40.315069 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba55e78e-2547-4f43-8464-dbebfbede211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:40.315957 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba55e78e-2547-4f43-8464-dbebfbede211 None admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:40.316063 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f70f19e-b134-45f1-a376-8e65dc18d375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:40.316366 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba55e78e-2547-4f43-8464-dbebfbede211 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:40.316618 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba55e78e-2547-4f43-8464-dbebfbede211 None admin] PATH_INFO: `/v3/projects/af60304e389b4295adfafbfc85459962/users/7fa722bf1210411e9101bfd161c43eaa/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:40.316722 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f70f19e-b134-45f1-a376-8e65dc18d375 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:40.316988 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f70f19e-b134-45f1-a376-8e65dc18d375 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:40.317266 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f70f19e-b134-45f1-a376-8e65dc18d375 None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:40.318012 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f70f19e-b134-45f1-a376-8e65dc18d375 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:40.318322 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f70f19e-b134-45f1-a376-8e65dc18d375 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:40.318641 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba55e78e-2547-4f43-8464-dbebfbede211 None admin] RBAC: Authorizing `identity:create_grant(project_id=af60304e389b4295adfafbfc85459962, user_id=7fa722bf1210411e9101bfd161c43eaa, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:40.319685 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f70f19e-b134-45f1-a376-8e65dc18d375 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:40.320057 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba55e78e-2547-4f43-8464-dbebfbede211 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:40.340274 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 67/126] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jun 15 14:30:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:40.340274 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f70f19e-b134-45f1-a376-8e65dc18d375 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:40.394829 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:40.398331 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 60/127] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Jun 15 14:30:40 2022] PUT /identity/v3/projects/af60304e389b4295adfafbfc85459962/users/7fa722bf1210411e9101bfd161c43eaa/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:30:40.399173 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba55e78e-2547-4f43-8464-dbebfbede211 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:40.584465 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f70f19e-b134-45f1-a376-8e65dc18d375 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:40.610236 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba55e78e-2547-4f43-8464-dbebfbede211 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:40.650988 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92392d97-389f-40fa-adcd-e175942ccb07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:40.656956 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92392d97-389f-40fa-adcd-e175942ccb07 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:40.659907 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92392d97-389f-40fa-adcd-e175942ccb07 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:40.660312 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92392d97-389f-40fa-adcd-e175942ccb07 None admin] PATH_INFO: `/v3/users/7fa722bf1210411e9101bfd161c43eaa` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:40.667871 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92392d97-389f-40fa-adcd-e175942ccb07 None admin] RBAC: Authorizing `identity:get_user(user_id=7fa722bf1210411e9101bfd161c43eaa)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:40.676163 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92392d97-389f-40fa-adcd-e175942ccb07 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:40.687481 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 68/128] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jun 15 14:30:40 2022] GET /identity/v3/users/7fa722bf1210411e9101bfd161c43eaa => generated 275 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:40.692920 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92392d97-389f-40fa-adcd-e175942ccb07 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:40.716566 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:40.774318 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b63d3ed-89a1-4458-9dc9-9ec3a7fe12fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:40.776987 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b63d3ed-89a1-4458-9dc9-9ec3a7fe12fc None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:40.777796 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b63d3ed-89a1-4458-9dc9-9ec3a7fe12fc None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:40.778729 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b63d3ed-89a1-4458-9dc9-9ec3a7fe12fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:40.783843 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b63d3ed-89a1-4458-9dc9-9ec3a7fe12fc None admin] RBAC: Adding query filter params (role.id=6ab04b221d1548c3b9c5941c17b9bd41, scope.project.id=af60304e389b4295adfafbfc85459962, user.id=aaa9a41e3d594efaaa8a9bd29ce1644e) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:40.785215 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b63d3ed-89a1-4458-9dc9-9ec3a7fe12fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:40.797145 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b63d3ed-89a1-4458-9dc9-9ec3a7fe12fc None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:40.811756 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 61/129] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jun 15 14:30:40 2022] GET /identity/v3/role_assignments?user.id=aaa9a41e3d594efaaa8a9bd29ce1644e&scope.project.id=af60304e389b4295adfafbfc85459962&role.id=6ab04b221d1548c3b9c5941c17b9bd41 => generated 259 bytes in 413 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:40.819991 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b63d3ed-89a1-4458-9dc9-9ec3a7fe12fc None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:41.024669 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92392d97-389f-40fa-adcd-e175942ccb07 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:41.046159 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:41.067172 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d276a3-9f5d-45c0-9cb2-61114adad4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:41.071653 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d276a3-9f5d-45c0-9cb2-61114adad4bf None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:41.071653 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d276a3-9f5d-45c0-9cb2-61114adad4bf None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:41.071653 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d276a3-9f5d-45c0-9cb2-61114adad4bf None admin] PATH_INFO: `/v3/users/7fa722bf1210411e9101bfd161c43eaa` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:41.071653 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d276a3-9f5d-45c0-9cb2-61114adad4bf None admin] RBAC: Authorizing `identity:get_user(user_id=7fa722bf1210411e9101bfd161c43eaa)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:41.073656 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d276a3-9f5d-45c0-9cb2-61114adad4bf None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:41.075818 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 69/130] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jun 15 14:30:40 2022] GET /identity/v3/users/7fa722bf1210411e9101bfd161c43eaa => generated 275 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:41.077461 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d276a3-9f5d-45c0-9cb2-61114adad4bf None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:41.123803 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b63d3ed-89a1-4458-9dc9-9ec3a7fe12fc None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:41.216013 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcd7ec1-da56-4db9-bf23-03c57f36b524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:41.219710 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcd7ec1-da56-4db9-bf23-03c57f36b524 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:41.221168 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcd7ec1-da56-4db9-bf23-03c57f36b524 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:41.221793 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcd7ec1-da56-4db9-bf23-03c57f36b524 None admin] PATH_INFO: `/v3/users/11d932a20a0f4d8b9ce5ee15655d936c` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:41.225783 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcd7ec1-da56-4db9-bf23-03c57f36b524 None admin] RBAC: Authorizing `identity:get_user(user_id=11d932a20a0f4d8b9ce5ee15655d936c)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:41.230425 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcd7ec1-da56-4db9-bf23-03c57f36b524 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:41.238227 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 62/131] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jun 15 14:30:40 2022] GET /identity/v3/users/11d932a20a0f4d8b9ce5ee15655d936c => generated 247 bytes in 426 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:41.240146 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcd7ec1-da56-4db9-bf23-03c57f36b524 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:41.345470 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d276a3-9f5d-45c0-9cb2-61114adad4bf None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:41.348461 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:41.402906 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46abf0cb-caca-4845-a7fd-23b1ef3948ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:41.404620 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46abf0cb-caca-4845-a7fd-23b1ef3948ce None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:41.405732 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46abf0cb-caca-4845-a7fd-23b1ef3948ce None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:41.406526 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46abf0cb-caca-4845-a7fd-23b1ef3948ce None admin] PATH_INFO: `/v3/projects/af60304e389b4295adfafbfc85459962` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:41.408832 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46abf0cb-caca-4845-a7fd-23b1ef3948ce None admin] RBAC: Authorizing `identity:get_project(project_id=af60304e389b4295adfafbfc85459962)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:41.411288 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46abf0cb-caca-4845-a7fd-23b1ef3948ce None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:41.414116 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 70/132] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jun 15 14:30:41 2022] GET /identity/v3/projects/af60304e389b4295adfafbfc85459962 => generated 298 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:41.415551 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d563c0f9-8b39-472b-bc2b-7eae2fdacc04 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:41.416049 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d563c0f9-8b39-472b-bc2b-7eae2fdacc04 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:41.416546 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d563c0f9-8b39-472b-bc2b-7eae2fdacc04 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:41.417954 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 71/133] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:30:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:41.419224 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d563c0f9-8b39-472b-bc2b-7eae2fdacc04 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:41.462836 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcd7ec1-da56-4db9-bf23-03c57f36b524 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:41.489412 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:41.515270 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9830798c-4f3d-4911-b4c3-c2506cd3928e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:41.516191 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830798c-4f3d-4911-b4c3-c2506cd3928e None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:41.516483 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830798c-4f3d-4911-b4c3-c2506cd3928e None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:41.516826 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830798c-4f3d-4911-b4c3-c2506cd3928e None admin] PATH_INFO: `/v3/projects/4cf55663d7eb45fd8ef9be9ec086db23` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:41.517996 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9830798c-4f3d-4911-b4c3-c2506cd3928e None admin] RBAC: Authorizing `identity:get_project(project_id=4cf55663d7eb45fd8ef9be9ec086db23)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:41.519441 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9830798c-4f3d-4911-b4c3-c2506cd3928e None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:41.521558 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 63/134] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jun 15 14:30:41 2022] GET /identity/v3/projects/4cf55663d7eb45fd8ef9be9ec086db23 => generated 312 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:41.522778 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd0dc38-eadb-4583-bcdf-4ce4e825a5a8 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:41.523078 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd0dc38-eadb-4583-bcdf-4ce4e825a5a8 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:41.523360 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd0dc38-eadb-4583-bcdf-4ce4e825a5a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:41.543045 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-cbd0dc38-eadb-4583-bcdf-4ce4e825a5a8 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:41.543045 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-cbd0dc38-eadb-4583-bcdf-4ce4e825a5a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:41.545020 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d563c0f9-8b39-472b-bc2b-7eae2fdacc04 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:41.587775 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120db21d-0fdc-47dc-ae81-e065db233715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:41.589269 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120db21d-0fdc-47dc-ae81-e065db233715 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:41.589776 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120db21d-0fdc-47dc-ae81-e065db233715 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:41.590288 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120db21d-0fdc-47dc-ae81-e065db233715 None admin] PATH_INFO: `/v3/projects/af60304e389b4295adfafbfc85459962` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:41.591846 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120db21d-0fdc-47dc-ae81-e065db233715 None admin] RBAC: Authorizing `identity:get_project(project_id=af60304e389b4295adfafbfc85459962)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:41.594499 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120db21d-0fdc-47dc-ae81-e065db233715 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:41.596717 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 72/135] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jun 15 14:30:41 2022] GET /identity/v3/projects/af60304e389b4295adfafbfc85459962 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:41.597806 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120db21d-0fdc-47dc-ae81-e065db233715 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:41.624178 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 64/136] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:30:41.626412 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd0dc38-eadb-4583-bcdf-4ce4e825a5a8 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:41.703727 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:41.710093 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120db21d-0fdc-47dc-ae81-e065db233715 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:41.741729 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:41.747262 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd0dc38-eadb-4583-bcdf-4ce4e825a5a8 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:41.755809 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a71772-36bd-47ad-a4a8-70a54ceaf7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:41.756904 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a71772-36bd-47ad-a4a8-70a54ceaf7a3 None admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:41.757741 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a71772-36bd-47ad-a4a8-70a54ceaf7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:41.758128 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a71772-36bd-47ad-a4a8-70a54ceaf7a3 None admin] PATH_INFO: `/v3/projects/af60304e389b4295adfafbfc85459962/users/7fa722bf1210411e9101bfd161c43eaa/roles/61e98f694ee741a9894b7e6db6dd11d3` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:41.761251 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a71772-36bd-47ad-a4a8-70a54ceaf7a3 None admin] RBAC: Authorizing `identity:create_grant(project_id=af60304e389b4295adfafbfc85459962, user_id=7fa722bf1210411e9101bfd161c43eaa, role_id=61e98f694ee741a9894b7e6db6dd11d3)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:41.763092 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a71772-36bd-47ad-a4a8-70a54ceaf7a3 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:41.786358 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45da867a-aa3d-42db-af4d-5caed6955941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:41.787248 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da867a-aa3d-42db-af4d-5caed6955941 None admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:41.787527 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da867a-aa3d-42db-af4d-5caed6955941 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:41.787802 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da867a-aa3d-42db-af4d-5caed6955941 None admin] PATH_INFO: `/v3/projects/4cf55663d7eb45fd8ef9be9ec086db23/users/7fa722bf1210411e9101bfd161c43eaa/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:41.789770 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45da867a-aa3d-42db-af4d-5caed6955941 None admin] RBAC: Authorizing `identity:create_grant(project_id=4cf55663d7eb45fd8ef9be9ec086db23, user_id=7fa722bf1210411e9101bfd161c43eaa, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:41.791205 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45da867a-aa3d-42db-af4d-5caed6955941 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:41.852403 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 73/137] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Jun 15 14:30:41 2022] PUT /identity/v3/projects/af60304e389b4295adfafbfc85459962/users/7fa722bf1210411e9101bfd161c43eaa/roles/61e98f694ee741a9894b7e6db6dd11d3 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:30:41.852403 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a71772-36bd-47ad-a4a8-70a54ceaf7a3 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:41.880557 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 65/138] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Jun 15 14:30:41 2022] PUT /identity/v3/projects/4cf55663d7eb45fd8ef9be9ec086db23/users/7fa722bf1210411e9101bfd161c43eaa/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:30:41.881472 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45da867a-aa3d-42db-af4d-5caed6955941 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:42.051343 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45da867a-aa3d-42db-af4d-5caed6955941 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:42.053892 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:42.080853 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a71772-36bd-47ad-a4a8-70a54ceaf7a3 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:42.094549 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f79866-408b-4f5a-8a9d-e871c107affc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:42.095377 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f79866-408b-4f5a-8a9d-e871c107affc None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:42.095662 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f79866-408b-4f5a-8a9d-e871c107affc None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:42.095938 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f79866-408b-4f5a-8a9d-e871c107affc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:42.097276 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f79866-408b-4f5a-8a9d-e871c107affc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:42.104187 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f79866-408b-4f5a-8a9d-e871c107affc None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:42.104187 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 66/139] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:42.107568 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f79866-408b-4f5a-8a9d-e871c107affc None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:42.159204 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9375f68-e116-425a-8529-b44777dbe9b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:42.160548 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9375f68-e116-425a-8529-b44777dbe9b0 None admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:42.161074 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9375f68-e116-425a-8529-b44777dbe9b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:42.161576 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9375f68-e116-425a-8529-b44777dbe9b0 None admin] PATH_INFO: `/v3/projects/af60304e389b4295adfafbfc85459962/users/11d932a20a0f4d8b9ce5ee15655d936c/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:42.163978 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9375f68-e116-425a-8529-b44777dbe9b0 None admin] RBAC: Authorizing `identity:create_grant(project_id=af60304e389b4295adfafbfc85459962, user_id=11d932a20a0f4d8b9ce5ee15655d936c, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:42.165770 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9375f68-e116-425a-8529-b44777dbe9b0 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:42.239710 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 74/140] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Jun 15 14:30:41 2022] PUT /identity/v3/projects/af60304e389b4295adfafbfc85459962/users/11d932a20a0f4d8b9ce5ee15655d936c/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:30:42.266134 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f79866-408b-4f5a-8a9d-e871c107affc None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:42.306437 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac16054e-4e6b-4a69-8628-21ec1b07dc33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:42.307489 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac16054e-4e6b-4a69-8628-21ec1b07dc33 None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:42.308024 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac16054e-4e6b-4a69-8628-21ec1b07dc33 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:42.308530 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac16054e-4e6b-4a69-8628-21ec1b07dc33 None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:42.309477 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac16054e-4e6b-4a69-8628-21ec1b07dc33 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:42.311345 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac16054e-4e6b-4a69-8628-21ec1b07dc33 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:42.434560 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 67/141] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Jun 15 14:30:42 2022] POST /identity/v3/users => generated 283 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:30:42.790573 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da63715-b03f-4e20-8994-4d76985a84f0 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:42.790573 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da63715-b03f-4e20-8994-4d76985a84f0 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:42.790573 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da63715-b03f-4e20-8994-4d76985a84f0 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:42.790573 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 75/142] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:30:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:42.798306 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62729b60-2e7b-4082-a916-00b20b39f1c5 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:42.798757 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62729b60-2e7b-4082-a916-00b20b39f1c5 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:42.798958 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62729b60-2e7b-4082-a916-00b20b39f1c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:42.837179 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-62729b60-2e7b-4082-a916-00b20b39f1c5 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:42.838430 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-62729b60-2e7b-4082-a916-00b20b39f1c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:42.864685 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:42.918032 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 68/143] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:30:42.944621 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da63715-b03f-4e20-8994-4d76985a84f0 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:43.012840 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:43.077981 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da63715-b03f-4e20-8994-4d76985a84f0 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:43.113905 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d11c09-83db-43b7-818b-970140da38d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:43.114716 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d11c09-83db-43b7-818b-970140da38d2 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:43.115001 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d11c09-83db-43b7-818b-970140da38d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:43.115283 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d11c09-83db-43b7-818b-970140da38d2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:43.127006 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d11c09-83db-43b7-818b-970140da38d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:30:43.134994 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-c1d11c09-83db-43b7-818b-970140da38d2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:30:43.134994 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:43.134994 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:43.134994 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:43.134994 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:43.134994 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:43.134994 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:43.134994 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:43.134994 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:43.134994 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:43.134994 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:43.134994 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:43.134994 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:30:43.134994 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:30:43.134994 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:30:43.134994 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:30:43.135874 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:30:43.135874 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:30:43.135874 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:30:43.135874 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:30:43.135874 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:43.135874 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:43.135874 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:43.135874 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:43.135874 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:43.135874 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:43.135874 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:43.135874 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:43.135874 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:43.135874 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:43.135874 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:43.135874 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:43.136693 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:43.136693 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:43.136693 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:43.136693 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:43.136693 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:30:43.136693 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:30:43.136693 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:30:43.136693 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:30:43.136693 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:30:43.136693 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:30:43.136693 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:30:43.136693 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:30:43.136693 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 76/144] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jun 15 14:30:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:30:43.145927 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62729b60-2e7b-4082-a916-00b20b39f1c5 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:43.318879 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62729b60-2e7b-4082-a916-00b20b39f1c5 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:43.373090 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1ea4d3-85bf-4224-8d01-c026a8c03cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:43.374524 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1ea4d3-85bf-4224-8d01-c026a8c03cb3 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:43.375015 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1ea4d3-85bf-4224-8d01-c026a8c03cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:43.375303 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1ea4d3-85bf-4224-8d01-c026a8c03cb3 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:43.376079 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1ea4d3-85bf-4224-8d01-c026a8c03cb3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:43.376421 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1ea4d3-85bf-4224-8d01-c026a8c03cb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:43.377818 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1ea4d3-85bf-4224-8d01-c026a8c03cb3 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:43.391836 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 69/145] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Jun 15 14:30:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:43.397500 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d11c09-83db-43b7-818b-970140da38d2 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:43.528327 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d11c09-83db-43b7-818b-970140da38d2 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:43.530068 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:43.577342 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00638430-8d99-4c94-ba41-b5e8b6e9a027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:43.578242 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00638430-8d99-4c94-ba41-b5e8b6e9a027 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:43.578527 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00638430-8d99-4c94-ba41-b5e8b6e9a027 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:43.578857 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00638430-8d99-4c94-ba41-b5e8b6e9a027 None admin] PATH_INFO: `/v3/users/7fa722bf1210411e9101bfd161c43eaa` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:43.580185 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00638430-8d99-4c94-ba41-b5e8b6e9a027 None admin] RBAC: Authorizing `identity:get_user(user_id=7fa722bf1210411e9101bfd161c43eaa)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:43.581656 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00638430-8d99-4c94-ba41-b5e8b6e9a027 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:43.583323 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 77/146] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jun 15 14:30:43 2022] GET /identity/v3/users/7fa722bf1210411e9101bfd161c43eaa => generated 275 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:43.589175 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1ea4d3-85bf-4224-8d01-c026a8c03cb3 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:43.629538 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:43.735728 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1ea4d3-85bf-4224-8d01-c026a8c03cb3 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:43.783740 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77708dd-1a75-44d9-8dfe-aa15638ed308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:43.784435 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77708dd-1a75-44d9-8dfe-aa15638ed308 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:43.784728 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77708dd-1a75-44d9-8dfe-aa15638ed308 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:43.785013 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77708dd-1a75-44d9-8dfe-aa15638ed308 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:43.810749 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5d5acd-ed6c-4dd5-a591-0956d933ef33 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:43.811443 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5d5acd-ed6c-4dd5-a591-0956d933ef33 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:43.811443 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5d5acd-ed6c-4dd5-a591-0956d933ef33 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:43.812906 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 78/147] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:30:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:43.821999 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b36e16-2a44-44f8-b1a1-eb38542894a3 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:43.822290 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b36e16-2a44-44f8-b1a1-eb38542894a3 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:43.822570 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b36e16-2a44-44f8-b1a1-eb38542894a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:43.850674 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-00b36e16-2a44-44f8-b1a1-eb38542894a3 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:43.851897 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-00b36e16-2a44-44f8-b1a1-eb38542894a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:43.914398 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77708dd-1a75-44d9-8dfe-aa15638ed308 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:43.915813 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77708dd-1a75-44d9-8dfe-aa15638ed308 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:43.943721 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:44.032697 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 79/148] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:30:44.052110 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 70/149] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jun 15 14:30:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:30:44.058322 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b36e16-2a44-44f8-b1a1-eb38542894a3 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:44.060302 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77708dd-1a75-44d9-8dfe-aa15638ed308 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:44.167838 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77708dd-1a75-44d9-8dfe-aa15638ed308 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:44.190041 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:44.214805 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0501f12b-2a14-43c1-b18c-66b8cedfb57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:44.215617 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0501f12b-2a14-43c1-b18c-66b8cedfb57b None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:44.215907 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0501f12b-2a14-43c1-b18c-66b8cedfb57b None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:44.216218 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0501f12b-2a14-43c1-b18c-66b8cedfb57b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:44.229816 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b36e16-2a44-44f8-b1a1-eb38542894a3 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:44.232480 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:44.243138 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0501f12b-2a14-43c1-b18c-66b8cedfb57b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:30:44.255553 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-0501f12b-2a14-43c1-b18c-66b8cedfb57b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:30:44.255553 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:44.255553 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:44.255553 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:44.255553 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:44.255553 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:44.255553 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:44.255553 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:44.255553 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:44.255553 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:44.255553 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:44.255553 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:44.255553 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:30:44.255553 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:30:44.255553 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:30:44.255553 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:30:44.256482 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:30:44.256482 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:30:44.256482 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:30:44.256482 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:30:44.256482 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:44.256482 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:44.256482 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:44.256482 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:44.256482 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:44.256482 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:44.256482 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:44.256482 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:44.256482 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:44.256482 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:44.256482 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:44.256482 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:44.257253 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:44.257253 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:44.257253 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:44.257253 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:44.257253 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:30:44.257253 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:30:44.257253 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:30:44.257253 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:30:44.257253 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:30:44.257253 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:30:44.257253 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:30:44.257253 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:30:44.257253 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 71/150] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jun 15 14:30:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:30:44.266146 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0501f12b-2a14-43c1-b18c-66b8cedfb57b None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:44.308586 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedb24cf-8172-47ec-9fac-d52bb92540a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:44.309452 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb24cf-8172-47ec-9fac-d52bb92540a1 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:44.309734 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb24cf-8172-47ec-9fac-d52bb92540a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:44.310010 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb24cf-8172-47ec-9fac-d52bb92540a1 None admin] PATH_INFO: `/v3/projects/af60304e389b4295adfafbfc85459962` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:44.312970 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedb24cf-8172-47ec-9fac-d52bb92540a1 None admin] RBAC: Authorizing `identity:get_project(project_id=af60304e389b4295adfafbfc85459962)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:44.314451 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedb24cf-8172-47ec-9fac-d52bb92540a1 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:44.318237 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 80/151] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jun 15 14:30:44 2022] GET /identity/v3/projects/af60304e389b4295adfafbfc85459962 => generated 298 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:44.322741 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedb24cf-8172-47ec-9fac-d52bb92540a1 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:44.486212 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0501f12b-2a14-43c1-b18c-66b8cedfb57b None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:44.553979 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedb24cf-8172-47ec-9fac-d52bb92540a1 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:44.577874 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4e2786-7ec0-488d-8f92-a27dde894828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:44.578554 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e2786-7ec0-488d-8f92-a27dde894828 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:44.578849 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e2786-7ec0-488d-8f92-a27dde894828 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:44.579129 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e2786-7ec0-488d-8f92-a27dde894828 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:44.579881 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4e2786-7ec0-488d-8f92-a27dde894828 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:44.580229 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4e2786-7ec0-488d-8f92-a27dde894828 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:44.581613 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4e2786-7ec0-488d-8f92-a27dde894828 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:44.601250 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 72/152] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Jun 15 14:30:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:44.606559 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4e2786-7ec0-488d-8f92-a27dde894828 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:44.628974 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da369f6-4347-4a65-b718-c6b4a50afc51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:44.629636 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da369f6-4347-4a65-b718-c6b4a50afc51 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:44.629916 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da369f6-4347-4a65-b718-c6b4a50afc51 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:44.630184 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da369f6-4347-4a65-b718-c6b4a50afc51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:44.631017 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da369f6-4347-4a65-b718-c6b4a50afc51 None admin] RBAC: Adding query filter params (role.id=9effaf496b874d01bd3f6d0555ef2d69, scope.project.id=af60304e389b4295adfafbfc85459962, user.id=7fa722bf1210411e9101bfd161c43eaa) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:44.631324 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da369f6-4347-4a65-b718-c6b4a50afc51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:44.632780 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da369f6-4347-4a65-b718-c6b4a50afc51 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:44.640818 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 81/153] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jun 15 14:30:44 2022] GET /identity/v3/role_assignments?user.id=7fa722bf1210411e9101bfd161c43eaa&scope.project.id=af60304e389b4295adfafbfc85459962&role.id=9effaf496b874d01bd3f6d0555ef2d69 => generated 608 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:44.698345 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:44.775477 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4e2786-7ec0-488d-8f92-a27dde894828 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:44.819065 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7f91fa-d71e-4663-9e56-c0f0aaaa9bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:44.819945 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7f91fa-d71e-4663-9e56-c0f0aaaa9bf5 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:44.820258 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7f91fa-d71e-4663-9e56-c0f0aaaa9bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:44.820537 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7f91fa-d71e-4663-9e56-c0f0aaaa9bf5 None admin] PATH_INFO: `/v3/users/aaa9a41e3d594efaaa8a9bd29ce1644e` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:44.821795 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7f91fa-d71e-4663-9e56-c0f0aaaa9bf5 None admin] RBAC: Authorizing `identity:get_user(user_id=aaa9a41e3d594efaaa8a9bd29ce1644e)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:44.823235 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7f91fa-d71e-4663-9e56-c0f0aaaa9bf5 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:44.825464 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 73/154] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jun 15 14:30:44 2022] GET /identity/v3/users/aaa9a41e3d594efaaa8a9bd29ce1644e => generated 289 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:44.830765 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da369f6-4347-4a65-b718-c6b4a50afc51 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:44.877231 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:44.965736 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da369f6-4347-4a65-b718-c6b4a50afc51 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:45.010705 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03cfd7c-892f-49d0-85c9-b961a1293e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:45.011588 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03cfd7c-892f-49d0-85c9-b961a1293e20 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:45.011869 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03cfd7c-892f-49d0-85c9-b961a1293e20 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:45.012166 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03cfd7c-892f-49d0-85c9-b961a1293e20 None admin] PATH_INFO: `/v3/projects/af60304e389b4295adfafbfc85459962` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:45.013374 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03cfd7c-892f-49d0-85c9-b961a1293e20 None admin] RBAC: Authorizing `identity:get_project(project_id=af60304e389b4295adfafbfc85459962)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:45.014901 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03cfd7c-892f-49d0-85c9-b961a1293e20 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:45.016661 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 82/155] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jun 15 14:30:44 2022] GET /identity/v3/projects/af60304e389b4295adfafbfc85459962 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:45.022767 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7f91fa-d71e-4663-9e56-c0f0aaaa9bf5 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:45.135441 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7f91fa-d71e-4663-9e56-c0f0aaaa9bf5 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:45.145859 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:45.174525 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054126ef-1a04-4728-91e9-e52f6c522f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:45.175593 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054126ef-1a04-4728-91e9-e52f6c522f53 None admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:45.175878 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054126ef-1a04-4728-91e9-e52f6c522f53 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:45.176185 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054126ef-1a04-4728-91e9-e52f6c522f53 None admin] PATH_INFO: `/v3/projects/af60304e389b4295adfafbfc85459962/users/aaa9a41e3d594efaaa8a9bd29ce1644e/roles/6ab04b221d1548c3b9c5941c17b9bd41` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:45.178166 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054126ef-1a04-4728-91e9-e52f6c522f53 None admin] RBAC: Authorizing `identity:create_grant(project_id=af60304e389b4295adfafbfc85459962, user_id=aaa9a41e3d594efaaa8a9bd29ce1644e, role_id=6ab04b221d1548c3b9c5941c17b9bd41)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:45.179617 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054126ef-1a04-4728-91e9-e52f6c522f53 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:45.268559 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 74/156] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Jun 15 14:30:45 2022] PUT /identity/v3/projects/af60304e389b4295adfafbfc85459962/users/aaa9a41e3d594efaaa8a9bd29ce1644e/roles/6ab04b221d1548c3b9c5941c17b9bd41 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:30:45.618168 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4d336-1ea3-4e6c-9cb3-ae277a81cc0b None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:45.618503 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4d336-1ea3-4e6c-9cb3-ae277a81cc0b None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:45.618725 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4d336-1ea3-4e6c-9cb3-ae277a81cc0b None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:45.620289 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 83/157] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:30:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:45.634747 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f72d644-3946-4816-968e-a4e5cf52d307 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:45.635035 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f72d644-3946-4816-968e-a4e5cf52d307 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:45.635310 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f72d644-3946-4816-968e-a4e5cf52d307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:45.661439 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-2f72d644-3946-4816-968e-a4e5cf52d307 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:45.662633 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-2f72d644-3946-4816-968e-a4e5cf52d307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:45.736486 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 75/158] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:30:45.774990 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b4d336-1ea3-4e6c-9cb3-ae277a81cc0b None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:45.893132 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:45.899954 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b4d336-1ea3-4e6c-9cb3-ae277a81cc0b None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:45.944243 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281c14c4-8d13-4a55-8086-bd614909bbe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:45.945063 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281c14c4-8d13-4a55-8086-bd614909bbe6 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:45.945342 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281c14c4-8d13-4a55-8086-bd614909bbe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:45.945615 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281c14c4-8d13-4a55-8086-bd614909bbe6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:45.961078 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281c14c4-8d13-4a55-8086-bd614909bbe6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:30:45.969992 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-281c14c4-8d13-4a55-8086-bd614909bbe6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:30:45.969992 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:45.969992 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:45.969992 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:45.969992 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:45.969992 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:45.969992 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:45.969992 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:45.969992 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:45.969992 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:45.969992 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:45.969992 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:45.969992 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:30:45.969992 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:30:45.969992 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:30:45.969992 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:30:45.970849 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:30:45.970849 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:30:45.970849 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:30:45.970849 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:30:45.970849 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:45.970849 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:45.970849 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:45.970849 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:45.970849 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:45.970849 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:45.970849 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:45.970849 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:45.970849 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:45.970849 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:45.970849 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:45.970849 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:45.971696 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:45.971696 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:45.971696 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:45.971696 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:45.971696 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:30:45.971696 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:30:45.971696 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:30:45.971696 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:30:45.971696 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:30:45.971696 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:30:45.971696 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:30:45.971696 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:30:45.971696 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 84/159] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Jun 15 14:30:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:45.979105 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f72d644-3946-4816-968e-a4e5cf52d307 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:46.124713 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:46.132848 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f72d644-3946-4816-968e-a4e5cf52d307 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:46.177511 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7656ac-a792-4b27-867e-15527909e2f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:46.178106 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7656ac-a792-4b27-867e-15527909e2f1 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:46.178389 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7656ac-a792-4b27-867e-15527909e2f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:46.178667 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7656ac-a792-4b27-867e-15527909e2f1 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:46.179422 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7656ac-a792-4b27-867e-15527909e2f1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:46.179731 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7656ac-a792-4b27-867e-15527909e2f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:46.181164 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7656ac-a792-4b27-867e-15527909e2f1 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:46.190712 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 76/160] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jun 15 14:30:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:46.196306 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281c14c4-8d13-4a55-8086-bd614909bbe6 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:46.221905 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b6f51-d93f-4582-b9b3-c3bea0dad356 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:46.222245 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b6f51-d93f-4582-b9b3-c3bea0dad356 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:46.222527 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b6f51-d93f-4582-b9b3-c3bea0dad356 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:46.224109 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 77/161] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:30:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:30:46.229141 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a502bd72-1596-4f4a-9289-531f0de7f9ba None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:46.229481 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a502bd72-1596-4f4a-9289-531f0de7f9ba None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:46.229766 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a502bd72-1596-4f4a-9289-531f0de7f9ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:46.261160 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-a502bd72-1596-4f4a-9289-531f0de7f9ba None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:46.262720 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a502bd72-1596-4f4a-9289-531f0de7f9ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:46.352347 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281c14c4-8d13-4a55-8086-bd614909bbe6 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:46.398131 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651fb5be-a577-437a-959a-e890e9f671bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:46.398996 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651fb5be-a577-437a-959a-e890e9f671bb None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:46.399275 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651fb5be-a577-437a-959a-e890e9f671bb None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:46.399552 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651fb5be-a577-437a-959a-e890e9f671bb None admin] PATH_INFO: `/v3/users/7fa722bf1210411e9101bfd161c43eaa` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:46.401467 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651fb5be-a577-437a-959a-e890e9f671bb None admin] RBAC: Authorizing `identity:get_user(user_id=7fa722bf1210411e9101bfd161c43eaa)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:46.403168 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651fb5be-a577-437a-959a-e890e9f671bb None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:46.405820 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 85/162] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jun 15 14:30:46 2022] GET /identity/v3/users/7fa722bf1210411e9101bfd161c43eaa => generated 275 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:46.407008 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e20f83-6d8e-4d26-8ab1-102a3c7fb279 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:46.407333 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e20f83-6d8e-4d26-8ab1-102a3c7fb279 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:46.407613 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e20f83-6d8e-4d26-8ab1-102a3c7fb279 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:46.409091 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 86/163] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:30:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:46.417645 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344ba1f1-1a10-4a33-acd6-2da9b6044743 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:46.417936 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344ba1f1-1a10-4a33-acd6-2da9b6044743 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:46.418212 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344ba1f1-1a10-4a33-acd6-2da9b6044743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:46.445883 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 78/164] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:30:46.447106 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a502bd72-1596-4f4a-9289-531f0de7f9ba None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:46.453058 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-344ba1f1-1a10-4a33-acd6-2da9b6044743 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:46.454283 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-344ba1f1-1a10-4a33-acd6-2da9b6044743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:46.492455 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:46.568995 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 87/165] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:30:46.569839 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344ba1f1-1a10-4a33-acd6-2da9b6044743 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:46.670333 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:46.684118 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a502bd72-1596-4f4a-9289-531f0de7f9ba None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:46.710015 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344ba1f1-1a10-4a33-acd6-2da9b6044743 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:46.735067 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31e0a32-b7cc-46c2-bd16-3cf2767d8f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:46.736026 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31e0a32-b7cc-46c2-bd16-3cf2767d8f58 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:46.736529 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31e0a32-b7cc-46c2-bd16-3cf2767d8f58 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:46.737063 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31e0a32-b7cc-46c2-bd16-3cf2767d8f58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:46.780974 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1ab2fa-f8bb-46f4-abe1-f76866694438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:46.781848 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1ab2fa-f8bb-46f4-abe1-f76866694438 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:46.782057 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1ab2fa-f8bb-46f4-abe1-f76866694438 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:46.782325 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1ab2fa-f8bb-46f4-abe1-f76866694438 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:46.798034 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1ab2fa-f8bb-46f4-abe1-f76866694438 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:30:46.815034 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-2b1ab2fa-f8bb-46f4-abe1-f76866694438 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:30:46.815034 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:46.815034 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:46.815034 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:46.815034 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:46.815034 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:46.815034 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:46.815034 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:46.815034 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:46.815034 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:46.815034 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:46.815034 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:46.815034 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:30:46.815034 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:30:46.815034 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:30:46.815034 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:30:46.815970 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:30:46.815970 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:30:46.815970 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:30:46.815970 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:30:46.815970 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:46.815970 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:46.815970 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:46.815970 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:46.815970 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:46.815970 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:46.815970 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:46.815970 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:46.815970 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:46.815970 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:46.815970 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:46.815970 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:46.816958 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:46.816958 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:46.816958 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:46.816958 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:46.816958 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:30:46.816958 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:30:46.816958 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:30:46.816958 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:30:46.816958 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:30:46.816958 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:30:46.816958 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:30:46.816958 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:30:46.824281 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 88/166] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jun 15 14:30:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:46.825241 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1ab2fa-f8bb-46f4-abe1-f76866694438 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:46.831803 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31e0a32-b7cc-46c2-bd16-3cf2767d8f58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:46.833581 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31e0a32-b7cc-46c2-bd16-3cf2767d8f58 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:46.849614 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:46.940030 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:46.953492 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 79/167] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jun 15 14:30:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:30:46.955972 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31e0a32-b7cc-46c2-bd16-3cf2767d8f58 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:47.086168 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1ab2fa-f8bb-46f4-abe1-f76866694438 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:47.090658 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31e0a32-b7cc-46c2-bd16-3cf2767d8f58 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:47.146704 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef00f52-d80e-4a75-b5cd-e8944a973020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:47.147491 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef00f52-d80e-4a75-b5cd-e8944a973020 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:47.147770 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef00f52-d80e-4a75-b5cd-e8944a973020 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:47.148046 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef00f52-d80e-4a75-b5cd-e8944a973020 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:47.149740 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f669b4-0795-43f8-8382-09ef66f4427e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:47.150648 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f669b4-0795-43f8-8382-09ef66f4427e None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:47.150922 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f669b4-0795-43f8-8382-09ef66f4427e None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:47.151191 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f669b4-0795-43f8-8382-09ef66f4427e None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:47.151928 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f669b4-0795-43f8-8382-09ef66f4427e None admin] RBAC: Adding query filter params (name=member) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:47.152257 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f669b4-0795-43f8-8382-09ef66f4427e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:47.153610 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f669b4-0795-43f8-8382-09ef66f4427e None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:47.168032 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 80/168] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Jun 15 14:30:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:47.169290 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121758c2-fb58-4eb2-b462-ec5f0debc046 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:47.169575 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121758c2-fb58-4eb2-b462-ec5f0debc046 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:47.169848 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121758c2-fb58-4eb2-b462-ec5f0debc046 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:47.173955 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef00f52-d80e-4a75-b5cd-e8944a973020 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:30:47.176803 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 81/169] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:30:47 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:47.177811 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121758c2-fb58-4eb2-b462-ec5f0debc046 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:47.181554 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-8ef00f52-d80e-4a75-b5cd-e8944a973020 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:30:47.181554 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:47.181554 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:47.181554 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:47.181554 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:47.181554 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:47.181554 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:47.181554 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:47.181554 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:47.181554 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:47.181554 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:47.181554 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:47.181554 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:30:47.181554 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:30:47.181554 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:30:47.181554 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:30:47.182504 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:30:47.182504 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:30:47.182504 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:30:47.182504 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:30:47.182504 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:47.182504 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:47.182504 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:47.182504 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:47.182504 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:47.182504 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:47.182504 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:47.182504 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:47.182504 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:47.182504 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:47.182504 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:47.182504 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:47.183586 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:47.183586 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:47.183586 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:47.183586 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:47.183586 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:30:47.183586 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:30:47.183586 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:30:47.183586 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:30:47.183586 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:30:47.183586 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:30:47.183586 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:30:47.183586 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:30:47.183586 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 89/170] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jun 15 14:30:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:47.183586 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef00f52-d80e-4a75-b5cd-e8944a973020 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:47.276427 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:47.323828 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:47.356649 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef00f52-d80e-4a75-b5cd-e8944a973020 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:47.359143 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121758c2-fb58-4eb2-b462-ec5f0debc046 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:47.422912 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22faae0d-2bff-46aa-92e9-5e15bd704c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:47.423798 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22faae0d-2bff-46aa-92e9-5e15bd704c93 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:47.424078 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22faae0d-2bff-46aa-92e9-5e15bd704c93 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:47.424374 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22faae0d-2bff-46aa-92e9-5e15bd704c93 None admin] PATH_INFO: `/v3/projects/af60304e389b4295adfafbfc85459962` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:47.428406 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22faae0d-2bff-46aa-92e9-5e15bd704c93 None admin] RBAC: Authorizing `identity:get_project(project_id=af60304e389b4295adfafbfc85459962)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:47.431795 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22faae0d-2bff-46aa-92e9-5e15bd704c93 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:47.433798 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 82/171] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jun 15 14:30:47 2022] GET /identity/v3/projects/af60304e389b4295adfafbfc85459962 => generated 298 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:47.434942 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde67658-b224-4b71-b91e-6498b2a26178 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:47.435228 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde67658-b224-4b71-b91e-6498b2a26178 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:47.435503 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde67658-b224-4b71-b91e-6498b2a26178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:47.448820 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56f95ac-3d7f-4a21-a2ca-a4a99d07789b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:47.450048 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56f95ac-3d7f-4a21-a2ca-a4a99d07789b None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:47.450546 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56f95ac-3d7f-4a21-a2ca-a4a99d07789b None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:47.451017 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56f95ac-3d7f-4a21-a2ca-a4a99d07789b None admin] PATH_INFO: `/v3/users/7fa722bf1210411e9101bfd161c43eaa` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:47.452605 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56f95ac-3d7f-4a21-a2ca-a4a99d07789b None admin] RBAC: Authorizing `identity:get_user(user_id=7fa722bf1210411e9101bfd161c43eaa)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:47.452717 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-fde67658-b224-4b71-b91e-6498b2a26178 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:47.453856 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-fde67658-b224-4b71-b91e-6498b2a26178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:47.454706 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56f95ac-3d7f-4a21-a2ca-a4a99d07789b None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:47.456618 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 90/172] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jun 15 14:30:47 2022] GET /identity/v3/users/7fa722bf1210411e9101bfd161c43eaa => generated 275 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:47.457637 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56f95ac-3d7f-4a21-a2ca-a4a99d07789b None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:47.544926 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 83/173] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:30:47.546436 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde67658-b224-4b71-b91e-6498b2a26178 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:47.591152 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56f95ac-3d7f-4a21-a2ca-a4a99d07789b None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:47.612148 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:47.634142 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da220bd2-68d0-49fe-8156-4b922b385b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:47.634818 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da220bd2-68d0-49fe-8156-4b922b385b5d None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:47.635102 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da220bd2-68d0-49fe-8156-4b922b385b5d None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:47.635375 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da220bd2-68d0-49fe-8156-4b922b385b5d None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:47.636338 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da220bd2-68d0-49fe-8156-4b922b385b5d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:47.636652 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da220bd2-68d0-49fe-8156-4b922b385b5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:47.638801 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da220bd2-68d0-49fe-8156-4b922b385b5d None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:47.650335 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 91/174] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jun 15 14:30:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:47.651150 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da220bd2-68d0-49fe-8156-4b922b385b5d None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:47.663967 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:47.675989 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde67658-b224-4b71-b91e-6498b2a26178 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:47.721948 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a653847-1b64-4e6b-8792-e4ff08cd4ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:47.723257 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a653847-1b64-4e6b-8792-e4ff08cd4ec6 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:47.723874 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a653847-1b64-4e6b-8792-e4ff08cd4ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:47.724423 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a653847-1b64-4e6b-8792-e4ff08cd4ec6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:47.726007 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a653847-1b64-4e6b-8792-e4ff08cd4ec6 None admin] RBAC: Adding query filter params (role.id=61e98f694ee741a9894b7e6db6dd11d3, scope.project.id=af60304e389b4295adfafbfc85459962, user.id=7fa722bf1210411e9101bfd161c43eaa) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:47.726534 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a653847-1b64-4e6b-8792-e4ff08cd4ec6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:47.728875 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a653847-1b64-4e6b-8792-e4ff08cd4ec6 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:47.742210 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 84/175] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jun 15 14:30:47 2022] GET /identity/v3/role_assignments?user.id=7fa722bf1210411e9101bfd161c43eaa&scope.project.id=af60304e389b4295adfafbfc85459962&role.id=61e98f694ee741a9894b7e6db6dd11d3 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:47.742210 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75585e96-10a2-4e43-955c-9be531eb27a3 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:47.742210 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75585e96-10a2-4e43-955c-9be531eb27a3 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:47.742210 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75585e96-10a2-4e43-955c-9be531eb27a3 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:47.742210 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 85/176] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:30:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:47.742210 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75585e96-10a2-4e43-955c-9be531eb27a3 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:47.810510 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da220bd2-68d0-49fe-8156-4b922b385b5d None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:47.881888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75585e96-10a2-4e43-955c-9be531eb27a3 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:47.899921 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:47.917211 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ce5836-a434-4a11-a8ef-b866b10eba8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:47.918090 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce5836-a434-4a11-a8ef-b866b10eba8c None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:47.918379 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce5836-a434-4a11-a8ef-b866b10eba8c None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:47.918657 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce5836-a434-4a11-a8ef-b866b10eba8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:47.926504 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabaa2c5-188b-4b96-bb34-f0b974359bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:47.927834 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabaa2c5-188b-4b96-bb34-f0b974359bff None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:47.928345 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabaa2c5-188b-4b96-bb34-f0b974359bff None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:47.928909 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabaa2c5-188b-4b96-bb34-f0b974359bff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:47.958976 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabaa2c5-188b-4b96-bb34-f0b974359bff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:30:47.966826 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-dabaa2c5-188b-4b96-bb34-f0b974359bff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:30:47.966826 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:47.966826 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:47.966826 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:47.966826 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:47.966826 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:47.966826 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:47.966826 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:47.966826 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:47.966826 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:47.966826 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:47.966826 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:47.966826 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:30:47.966826 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:30:47.966826 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:30:47.966826 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:30:47.968725 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:30:47.968725 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:30:47.968725 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:30:47.968725 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:30:47.968725 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:47.968725 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:47.968725 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:47.968725 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:47.968725 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:47.968725 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:47.968725 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:47.968725 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:47.968725 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:47.968725 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:47.968725 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:47.968725 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:47.969502 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:47.969502 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:47.969502 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:47.969502 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:47.969502 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:30:47.969502 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:30:47.969502 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:30:47.969502 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:30:47.969502 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:30:47.969502 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:30:47.969502 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:30:47.969502 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:30:47.970087 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 86/177] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jun 15 14:30:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:47.970924 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabaa2c5-188b-4b96-bb34-f0b974359bff None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:48.028459 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ce5836-a434-4a11-a8ef-b866b10eba8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:48.030015 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ce5836-a434-4a11-a8ef-b866b10eba8c None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:48.042969 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:48.135190 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabaa2c5-188b-4b96-bb34-f0b974359bff None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:48.177006 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcf05d8-bfd9-4611-8736-90f533dc043b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:48.177885 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf05d8-bfd9-4611-8736-90f533dc043b None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:48.178391 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf05d8-bfd9-4611-8736-90f533dc043b None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:48.178514 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 92/178] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jun 15 14:30:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 527 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:30:48.178858 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf05d8-bfd9-4611-8736-90f533dc043b None admin] PATH_INFO: `/v3/users/11d932a20a0f4d8b9ce5ee15655d936c` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:48.181877 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe405c64-9a12-47ee-b78e-d26096f7751c None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:48.181989 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcf05d8-bfd9-4611-8736-90f533dc043b None admin] RBAC: Authorizing `identity:get_user(user_id=11d932a20a0f4d8b9ce5ee15655d936c)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:48.182062 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe405c64-9a12-47ee-b78e-d26096f7751c None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:48.182062 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe405c64-9a12-47ee-b78e-d26096f7751c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:48.182203 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcf05d8-bfd9-4611-8736-90f533dc043b None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:48.182990 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 87/179] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jun 15 14:30:47 2022] GET /identity/v3/users/11d932a20a0f4d8b9ce5ee15655d936c => generated 247 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:48.183673 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcf05d8-bfd9-4611-8736-90f533dc043b None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:48.198372 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-fe405c64-9a12-47ee-b78e-d26096f7751c None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:48.199587 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-fe405c64-9a12-47ee-b78e-d26096f7751c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:48.235015 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:48.268399 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 93/180] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:30:48.269772 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8c4ef1-8d53-4f59-ba59-b1d519392001 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:48.270066 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8c4ef1-8d53-4f59-ba59-b1d519392001 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:48.270339 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8c4ef1-8d53-4f59-ba59-b1d519392001 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:48.271818 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 94/181] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:30:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:48.272411 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8c4ef1-8d53-4f59-ba59-b1d519392001 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:48.302652 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcf05d8-bfd9-4611-8736-90f533dc043b None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:48.319303 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:48.338389 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1ae528-6839-4e18-8334-4f7951490aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:48.339236 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ae528-6839-4e18-8334-4f7951490aa6 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:48.339538 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ae528-6839-4e18-8334-4f7951490aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:48.339805 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ae528-6839-4e18-8334-4f7951490aa6 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:48.340684 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1ae528-6839-4e18-8334-4f7951490aa6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:48.340996 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1ae528-6839-4e18-8334-4f7951490aa6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:48.342902 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1ae528-6839-4e18-8334-4f7951490aa6 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:48.351515 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 88/182] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jun 15 14:30:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:48.356933 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1ae528-6839-4e18-8334-4f7951490aa6 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:48.388543 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8c4ef1-8d53-4f59-ba59-b1d519392001 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:48.429164 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd2a71f-ea5b-46da-af90-149251626ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:48.430032 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd2a71f-ea5b-46da-af90-149251626ed0 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:48.430314 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd2a71f-ea5b-46da-af90-149251626ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:48.430592 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd2a71f-ea5b-46da-af90-149251626ed0 None admin] PATH_INFO: `/v3/projects/4cf55663d7eb45fd8ef9be9ec086db23` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:48.431864 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd2a71f-ea5b-46da-af90-149251626ed0 None admin] RBAC: Authorizing `identity:get_project(project_id=4cf55663d7eb45fd8ef9be9ec086db23)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:48.433432 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd2a71f-ea5b-46da-af90-149251626ed0 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:48.435298 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 95/183] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jun 15 14:30:48 2022] GET /identity/v3/projects/4cf55663d7eb45fd8ef9be9ec086db23 => generated 312 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:48.437415 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9815f0c7-446d-4a1f-92ed-bf2c7128abe5 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:48.437758 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9815f0c7-446d-4a1f-92ed-bf2c7128abe5 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:48.438062 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9815f0c7-446d-4a1f-92ed-bf2c7128abe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:48.455359 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-9815f0c7-446d-4a1f-92ed-bf2c7128abe5 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:48.456581 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-9815f0c7-446d-4a1f-92ed-bf2c7128abe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:48.465032 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1ae528-6839-4e18-8334-4f7951490aa6 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:48.467868 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:48.498829 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff1e2e2-1900-49e6-a6e8-9cd938c4a0f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:48.499487 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff1e2e2-1900-49e6-a6e8-9cd938c4a0f1 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:48.499765 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff1e2e2-1900-49e6-a6e8-9cd938c4a0f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:48.500038 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff1e2e2-1900-49e6-a6e8-9cd938c4a0f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:48.523198 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 96/184] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:30:48.524022 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9815f0c7-446d-4a1f-92ed-bf2c7128abe5 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:48.562933 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:48.581359 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff1e2e2-1900-49e6-a6e8-9cd938c4a0f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:48.582851 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff1e2e2-1900-49e6-a6e8-9cd938c4a0f1 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:48.629343 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9815f0c7-446d-4a1f-92ed-bf2c7128abe5 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:48.664662 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d80b1-5b0c-4c5b-b4ae-35702e9f7f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:48.665385 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d80b1-5b0c-4c5b-b4ae-35702e9f7f1c None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:48.665667 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d80b1-5b0c-4c5b-b4ae-35702e9f7f1c None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:48.665944 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d80b1-5b0c-4c5b-b4ae-35702e9f7f1c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:48.675767 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71d80b1-5b0c-4c5b-b4ae-35702e9f7f1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:30:48.679477 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 89/185] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jun 15 14:30:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:30:48.680328 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff1e2e2-1900-49e6-a6e8-9cd938c4a0f1 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:48.683889 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-d71d80b1-5b0c-4c5b-b4ae-35702e9f7f1c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:30:48.683889 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:48.683889 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:48.683889 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:48.683889 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:48.683889 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:48.683889 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:48.683889 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:48.683889 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:48.683889 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:48.683889 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:48.683889 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:48.683889 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:30:48.683889 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:30:48.683889 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:30:48.683889 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:30:48.684805 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:30:48.684805 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:30:48.684805 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:30:48.684805 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:30:48.684805 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:48.684805 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:48.684805 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:48.684805 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:48.684805 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:48.684805 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:48.684805 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:48.684805 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:48.684805 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:48.684805 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:48.684805 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:48.684805 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:48.685587 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:48.685587 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:48.685587 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:48.685587 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:48.685587 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:30:48.685587 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:30:48.685587 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:30:48.685587 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:30:48.685587 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:30:48.685587 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:30:48.685587 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:30:48.685587 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:30:48.686604 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:48.686673 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 97/186] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Jun 15 14:30:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:48.686673 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d80b1-5b0c-4c5b-b4ae-35702e9f7f1c None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:48.781241 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:48.782651 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff1e2e2-1900-49e6-a6e8-9cd938c4a0f1 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:48.786876 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d80b1-5b0c-4c5b-b4ae-35702e9f7f1c None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:48.816293 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86d6db1-34ef-4c90-be83-031dfcfeb6f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:48.817153 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86d6db1-34ef-4c90-be83-031dfcfeb6f1 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:48.817434 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86d6db1-34ef-4c90-be83-031dfcfeb6f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:48.817706 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86d6db1-34ef-4c90-be83-031dfcfeb6f1 None admin] PATH_INFO: `/v3/users/fe1e5bd379ef4b7ab3ea48ca4c9923ab` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:48.819897 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944726e9-0efb-40fe-a7de-61f5175cf215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:48.820590 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944726e9-0efb-40fe-a7de-61f5175cf215 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:48.820876 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944726e9-0efb-40fe-a7de-61f5175cf215 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:48.821155 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944726e9-0efb-40fe-a7de-61f5175cf215 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:48.822032 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944726e9-0efb-40fe-a7de-61f5175cf215 None admin] RBAC: Adding query filter params (role.id=9effaf496b874d01bd3f6d0555ef2d69, scope.project.id=4cf55663d7eb45fd8ef9be9ec086db23, user.id=7fa722bf1210411e9101bfd161c43eaa) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:48.822337 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944726e9-0efb-40fe-a7de-61f5175cf215 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:48.823812 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944726e9-0efb-40fe-a7de-61f5175cf215 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:48.830449 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 98/187] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jun 15 14:30:48 2022] GET /identity/v3/role_assignments?user.id=7fa722bf1210411e9101bfd161c43eaa&scope.project.id=4cf55663d7eb45fd8ef9be9ec086db23&role.id=9effaf496b874d01bd3f6d0555ef2d69 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:48.831663 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c827ef78-d0de-437e-b448-175855b0246f None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:48.831950 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c827ef78-d0de-437e-b448-175855b0246f None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:48.832267 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c827ef78-d0de-437e-b448-175855b0246f None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:48.833355 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86d6db1-34ef-4c90-be83-031dfcfeb6f1 None admin] RBAC: Authorizing `identity:get_user(user_id=fe1e5bd379ef4b7ab3ea48ca4c9923ab)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:48.833539 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 99/188] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:30:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:48.834210 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c827ef78-d0de-437e-b448-175855b0246f None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:48.834839 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86d6db1-34ef-4c90-be83-031dfcfeb6f1 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:48.836930 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 90/189] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jun 15 14:30:48 2022] GET /identity/v3/users/fe1e5bd379ef4b7ab3ea48ca4c9923ab => generated 283 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:48.844163 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86d6db1-34ef-4c90-be83-031dfcfeb6f1 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:48.916721 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:48.951114 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c827ef78-d0de-437e-b448-175855b0246f None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:48.964296 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86d6db1-34ef-4c90-be83-031dfcfeb6f1 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:48.997702 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a87a61-b400-44a8-a218-a220021a4401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:48.999184 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a87a61-b400-44a8-a218-a220021a4401 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:48.999517 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a87a61-b400-44a8-a218-a220021a4401 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:48.999797 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a87a61-b400-44a8-a218-a220021a4401 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:49.000648 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123e6a94-aa19-4e09-bbb5-4c72658d02a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:49.001037 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a87a61-b400-44a8-a218-a220021a4401 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:49.001512 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123e6a94-aa19-4e09-bbb5-4c72658d02a3 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:49.001792 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123e6a94-aa19-4e09-bbb5-4c72658d02a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:49.002066 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123e6a94-aa19-4e09-bbb5-4c72658d02a3 None admin] PATH_INFO: `/v3/projects/af60304e389b4295adfafbfc85459962` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:49.002485 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a87a61-b400-44a8-a218-a220021a4401 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:49.003288 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123e6a94-aa19-4e09-bbb5-4c72658d02a3 None admin] RBAC: Authorizing `identity:get_project(project_id=af60304e389b4295adfafbfc85459962)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:49.004120 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 100/190] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:49.004753 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123e6a94-aa19-4e09-bbb5-4c72658d02a3 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:49.004855 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a87a61-b400-44a8-a218-a220021a4401 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:49.006561 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 91/191] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jun 15 14:30:48 2022] GET /identity/v3/projects/af60304e389b4295adfafbfc85459962 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:49.007182 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123e6a94-aa19-4e09-bbb5-4c72658d02a3 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:49.042318 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:49.119521 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a87a61-b400-44a8-a218-a220021a4401 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:49.120354 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123e6a94-aa19-4e09-bbb5-4c72658d02a3 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:49.148340 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:49.153221 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac48dbb8-1641-4af8-86bc-8d4936cf6a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:49.153889 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac48dbb8-1641-4af8-86bc-8d4936cf6a75 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:49.154139 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64330f28-1a59-4873-b0da-4736152e47c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:49.154248 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac48dbb8-1641-4af8-86bc-8d4936cf6a75 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:49.154442 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac48dbb8-1641-4af8-86bc-8d4936cf6a75 None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:49.154956 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64330f28-1a59-4873-b0da-4736152e47c9 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:49.155197 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac48dbb8-1641-4af8-86bc-8d4936cf6a75 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:49.155391 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64330f28-1a59-4873-b0da-4736152e47c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:49.155483 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac48dbb8-1641-4af8-86bc-8d4936cf6a75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:49.155676 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64330f28-1a59-4873-b0da-4736152e47c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:49.156879 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac48dbb8-1641-4af8-86bc-8d4936cf6a75 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:49.165371 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 101/192] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jun 15 14:30:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:49.166813 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b438fe-3a19-4a52-b13f-89747a95ddef None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:49.167041 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b438fe-3a19-4a52-b13f-89747a95ddef None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:49.167314 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b438fe-3a19-4a52-b13f-89747a95ddef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:49.184518 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-75b438fe-3a19-4a52-b13f-89747a95ddef None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:49.186176 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-75b438fe-3a19-4a52-b13f-89747a95ddef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:49.223217 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64330f28-1a59-4873-b0da-4736152e47c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:49.224712 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64330f28-1a59-4873-b0da-4736152e47c9 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:49.251873 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 102/193] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:30:49.252872 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b438fe-3a19-4a52-b13f-89747a95ddef None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:49.274440 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:49.325739 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 92/194] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jun 15 14:30:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:30:49.326695 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64330f28-1a59-4873-b0da-4736152e47c9 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:49.354567 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:49.358898 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b438fe-3a19-4a52-b13f-89747a95ddef None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:49.392530 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95018ab-c32b-4c44-b0a5-61b8274eaed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:49.392995 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95018ab-c32b-4c44-b0a5-61b8274eaed2 None admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:49.393272 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95018ab-c32b-4c44-b0a5-61b8274eaed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:49.393547 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95018ab-c32b-4c44-b0a5-61b8274eaed2 None admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:49.394250 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95018ab-c32b-4c44-b0a5-61b8274eaed2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:49.395685 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95018ab-c32b-4c44-b0a5-61b8274eaed2 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:49.424930 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64330f28-1a59-4873-b0da-4736152e47c9 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:49.458683 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7567211f-de20-45af-8031-80121fb11ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:49.459330 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7567211f-de20-45af-8031-80121fb11ca6 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:49.459611 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7567211f-de20-45af-8031-80121fb11ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:49.459884 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7567211f-de20-45af-8031-80121fb11ca6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:49.460743 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7567211f-de20-45af-8031-80121fb11ca6 None admin] RBAC: Adding query filter params (role.id=2bd1979f53bb4853a50dd5bee6aa0782, scope.project.id=af60304e389b4295adfafbfc85459962, user.id=11d932a20a0f4d8b9ce5ee15655d936c) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:49.461056 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7567211f-de20-45af-8031-80121fb11ca6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:49.462479 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7567211f-de20-45af-8031-80121fb11ca6 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:49.467870 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 103/195] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Jun 15 14:30:49 2022] POST /identity/v3/users => generated 297 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:30:49.468197 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 93/196] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jun 15 14:30:49 2022] GET /identity/v3/role_assignments?user.id=11d932a20a0f4d8b9ce5ee15655d936c&scope.project.id=af60304e389b4295adfafbfc85459962&role.id=2bd1979f53bb4853a50dd5bee6aa0782 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:49.468724 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95018ab-c32b-4c44-b0a5-61b8274eaed2 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:49.468962 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7567211f-de20-45af-8031-80121fb11ca6 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:49.574784 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:49.611972 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95018ab-c32b-4c44-b0a5-61b8274eaed2 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:49.620717 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:49.631608 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7567211f-de20-45af-8031-80121fb11ca6 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:49.659041 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5ff2cc-5dcc-4c71-add5-d959462f1ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:49.659326 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5ff2cc-5dcc-4c71-add5-d959462f1ab1 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:49.659618 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5ff2cc-5dcc-4c71-add5-d959462f1ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:49.660762 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5ff2cc-5dcc-4c71-add5-d959462f1ab1 None admin] PATH_INFO: `/v3/users/fe1e5bd379ef4b7ab3ea48ca4c9923ab` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:49.661453 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5ff2cc-5dcc-4c71-add5-d959462f1ab1 None admin] RBAC: Authorizing `identity:get_user(user_id=fe1e5bd379ef4b7ab3ea48ca4c9923ab)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:49.662939 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5ff2cc-5dcc-4c71-add5-d959462f1ab1 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:49.664998 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 104/197] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jun 15 14:30:49 2022] GET /identity/v3/users/fe1e5bd379ef4b7ab3ea48ca4c9923ab => generated 283 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:49.665948 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5ff2cc-5dcc-4c71-add5-d959462f1ab1 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:49.687919 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966af504-b76a-4e82-8e2a-4818dac016f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:49.689808 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966af504-b76a-4e82-8e2a-4818dac016f3 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:49.689808 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966af504-b76a-4e82-8e2a-4818dac016f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:49.689808 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966af504-b76a-4e82-8e2a-4818dac016f3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:49.705299 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966af504-b76a-4e82-8e2a-4818dac016f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:30:49.715207 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-966af504-b76a-4e82-8e2a-4818dac016f3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:30:49.715207 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:49.715207 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:49.715207 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:49.715207 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:49.715207 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:49.715207 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:49.715207 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:49.715207 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:49.715207 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:49.715207 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:49.715207 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:49.715207 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:30:49.715207 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:30:49.715207 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:30:49.715207 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:30:49.716404 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:30:49.716404 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:30:49.716404 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:30:49.716404 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:30:49.716404 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:49.716404 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:49.716404 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:49.716404 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:49.716404 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:49.716404 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:49.716404 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:49.716404 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:49.716404 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:49.716404 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:49.716404 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:49.716404 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:49.717881 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:49.717881 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:49.717881 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:49.717881 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:49.717881 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:30:49.717881 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:30:49.717881 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:30:49.717881 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:30:49.717881 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:30:49.717881 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:30:49.717881 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:30:49.717881 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:30:49.720334 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 94/198] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jun 15 14:30:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:49.722552 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966af504-b76a-4e82-8e2a-4818dac016f3 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:49.800379 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5ff2cc-5dcc-4c71-add5-d959462f1ab1 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:49.839960 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78e3e4c-9772-4e3c-86a3-10a246e34760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:49.840895 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e3e4c-9772-4e3c-86a3-10a246e34760 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:49.841620 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e3e4c-9772-4e3c-86a3-10a246e34760 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:49.841931 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e3e4c-9772-4e3c-86a3-10a246e34760 None admin] PATH_INFO: `/v3/projects/e58ea743785943a9936197cc15d05a80` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:49.843753 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78e3e4c-9772-4e3c-86a3-10a246e34760 None admin] RBAC: Authorizing `identity:get_project(project_id=e58ea743785943a9936197cc15d05a80)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:49.845747 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78e3e4c-9772-4e3c-86a3-10a246e34760 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:49.846993 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 105/199] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jun 15 14:30:49 2022] GET /identity/v3/projects/e58ea743785943a9936197cc15d05a80 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:49.848751 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78e3e4c-9772-4e3c-86a3-10a246e34760 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:49.856046 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966af504-b76a-4e82-8e2a-4818dac016f3 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:49.865654 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:49.901939 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5250497-8ec9-4b2a-b1db-7dc6f61a8222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:49.902596 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5250497-8ec9-4b2a-b1db-7dc6f61a8222 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:49.902882 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5250497-8ec9-4b2a-b1db-7dc6f61a8222 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:49.903159 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5250497-8ec9-4b2a-b1db-7dc6f61a8222 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:49.936013 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:49.983796 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78e3e4c-9772-4e3c-86a3-10a246e34760 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:49.983980 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5250497-8ec9-4b2a-b1db-7dc6f61a8222 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:49.985420 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5250497-8ec9-4b2a-b1db-7dc6f61a8222 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:50.027437 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94580b07-25a3-4f80-a831-0aa4b3365ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:50.028149 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94580b07-25a3-4f80-a831-0aa4b3365ee3 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:50.028492 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94580b07-25a3-4f80-a831-0aa4b3365ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:50.028779 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94580b07-25a3-4f80-a831-0aa4b3365ee3 None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:50.029570 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94580b07-25a3-4f80-a831-0aa4b3365ee3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:50.029877 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94580b07-25a3-4f80-a831-0aa4b3365ee3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:50.031284 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94580b07-25a3-4f80-a831-0aa4b3365ee3 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:50.040338 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 106/200] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Jun 15 14:30:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:50.045012 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94580b07-25a3-4f80-a831-0aa4b3365ee3 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:50.106682 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 95/201] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jun 15 14:30:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:30:50.112748 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5250497-8ec9-4b2a-b1db-7dc6f61a8222 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:50.140308 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:50.167495 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94580b07-25a3-4f80-a831-0aa4b3365ee3 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:50.172672 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:50.217053 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbc1f96-ac7c-43e7-9172-6ce3429c36af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:50.218733 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbc1f96-ac7c-43e7-9172-6ce3429c36af None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:50.219159 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbc1f96-ac7c-43e7-9172-6ce3429c36af None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:50.219808 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbc1f96-ac7c-43e7-9172-6ce3429c36af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:50.221237 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbc1f96-ac7c-43e7-9172-6ce3429c36af None admin] RBAC: Adding query filter params (role.id=2bd1979f53bb4853a50dd5bee6aa0782, scope.project.id=e58ea743785943a9936197cc15d05a80, user.id=fe1e5bd379ef4b7ab3ea48ca4c9923ab) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:50.221968 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbc1f96-ac7c-43e7-9172-6ce3429c36af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:50.223566 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbc1f96-ac7c-43e7-9172-6ce3429c36af None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:50.223744 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5250497-8ec9-4b2a-b1db-7dc6f61a8222 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:50.231037 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 107/202] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jun 15 14:30:50 2022] GET /identity/v3/role_assignments?user.id=fe1e5bd379ef4b7ab3ea48ca4c9923ab&scope.project.id=e58ea743785943a9936197cc15d05a80&role.id=2bd1979f53bb4853a50dd5bee6aa0782 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:50.233115 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbc1f96-ac7c-43e7-9172-6ce3429c36af None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:50.267968 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8051f8a4-d2a8-4cc1-a73c-bb6ab8ebae2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:50.269110 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8051f8a4-d2a8-4cc1-a73c-bb6ab8ebae2e None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:50.269490 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8051f8a4-d2a8-4cc1-a73c-bb6ab8ebae2e None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:50.269885 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8051f8a4-d2a8-4cc1-a73c-bb6ab8ebae2e None admin] PATH_INFO: `/v3/users/aaa9a41e3d594efaaa8a9bd29ce1644e` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:50.271875 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8051f8a4-d2a8-4cc1-a73c-bb6ab8ebae2e None admin] RBAC: Authorizing `identity:get_user(user_id=aaa9a41e3d594efaaa8a9bd29ce1644e)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:50.273471 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8051f8a4-d2a8-4cc1-a73c-bb6ab8ebae2e None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:50.275811 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 96/203] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jun 15 14:30:50 2022] GET /identity/v3/users/aaa9a41e3d594efaaa8a9bd29ce1644e => generated 289 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:50.282540 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8051f8a4-d2a8-4cc1-a73c-bb6ab8ebae2e None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:50.368612 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbc1f96-ac7c-43e7-9172-6ce3429c36af None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:50.413209 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fb1087-61eb-4045-b1ef-680ff732a9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:50.414576 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb1087-61eb-4045-b1ef-680ff732a9f9 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:50.415442 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb1087-61eb-4045-b1ef-680ff732a9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:50.415862 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb1087-61eb-4045-b1ef-680ff732a9f9 None admin] PATH_INFO: `/v3/projects/e58ea743785943a9936197cc15d05a80` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:50.417410 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fb1087-61eb-4045-b1ef-680ff732a9f9 None admin] RBAC: Authorizing `identity:get_project(project_id=e58ea743785943a9936197cc15d05a80)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:50.419126 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fb1087-61eb-4045-b1ef-680ff732a9f9 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:50.421487 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 108/204] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jun 15 14:30:50 2022] GET /identity/v3/projects/e58ea743785943a9936197cc15d05a80 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:50.428845 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fb1087-61eb-4045-b1ef-680ff732a9f9 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:50.460484 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:50.461707 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8051f8a4-d2a8-4cc1-a73c-bb6ab8ebae2e None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:50.488823 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:50.501375 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0701a4d-3575-4a1f-9abf-57d9e629a545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:50.504118 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0701a4d-3575-4a1f-9abf-57d9e629a545 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:50.504118 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0701a4d-3575-4a1f-9abf-57d9e629a545 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:50.504118 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0701a4d-3575-4a1f-9abf-57d9e629a545 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:50.577658 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0701a4d-3575-4a1f-9abf-57d9e629a545 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:50.579212 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0701a4d-3575-4a1f-9abf-57d9e629a545 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:50.588763 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fb1087-61eb-4045-b1ef-680ff732a9f9 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:50.661557 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad26f10-752e-42d3-a52f-ce4554ce1747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:50.662219 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad26f10-752e-42d3-a52f-ce4554ce1747 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:50.662507 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad26f10-752e-42d3-a52f-ce4554ce1747 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:50.662789 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad26f10-752e-42d3-a52f-ce4554ce1747 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:50.663743 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad26f10-752e-42d3-a52f-ce4554ce1747 None admin] RBAC: Adding query filter params (role.id=61e98f694ee741a9894b7e6db6dd11d3, scope.project.id=e58ea743785943a9936197cc15d05a80, user.id=fe1e5bd379ef4b7ab3ea48ca4c9923ab) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:50.664066 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad26f10-752e-42d3-a52f-ce4554ce1747 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:50.665535 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad26f10-752e-42d3-a52f-ce4554ce1747 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:50.675478 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 109/205] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jun 15 14:30:50 2022] GET /identity/v3/role_assignments?user.id=fe1e5bd379ef4b7ab3ea48ca4c9923ab&scope.project.id=e58ea743785943a9936197cc15d05a80&role.id=61e98f694ee741a9894b7e6db6dd11d3 => generated 259 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:50.763423 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 97/206] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jun 15 14:30:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:30:50.769655 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad26f10-752e-42d3-a52f-ce4554ce1747 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:50.880901 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:50.924593 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad26f10-752e-42d3-a52f-ce4554ce1747 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:50.964812 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2349388a-a1a6-495d-ab5e-cadc67612216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:50.965753 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2349388a-a1a6-495d-ab5e-cadc67612216 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:50.966037 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2349388a-a1a6-495d-ab5e-cadc67612216 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:50.966309 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2349388a-a1a6-495d-ab5e-cadc67612216 None admin] PATH_INFO: `/v3/projects/af60304e389b4295adfafbfc85459962` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:50.968178 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2349388a-a1a6-495d-ab5e-cadc67612216 None admin] RBAC: Authorizing `identity:get_project(project_id=af60304e389b4295adfafbfc85459962)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:50.969682 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2349388a-a1a6-495d-ab5e-cadc67612216 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:50.971407 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 110/207] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jun 15 14:30:50 2022] GET /identity/v3/projects/af60304e389b4295adfafbfc85459962 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:50.977525 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0701a4d-3575-4a1f-9abf-57d9e629a545 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:51.070928 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:51.180815 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0701a4d-3575-4a1f-9abf-57d9e629a545 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:51.227607 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c5a751-5405-4b37-8e71-114f956a6d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:51.228370 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c5a751-5405-4b37-8e71-114f956a6d09 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:51.228573 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c5a751-5405-4b37-8e71-114f956a6d09 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:51.228855 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c5a751-5405-4b37-8e71-114f956a6d09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:51.229697 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c5a751-5405-4b37-8e71-114f956a6d09 None admin] RBAC: Adding query filter params (role.id=6ab04b221d1548c3b9c5941c17b9bd41, scope.project.id=af60304e389b4295adfafbfc85459962, user.id=aaa9a41e3d594efaaa8a9bd29ce1644e) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:51.230016 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c5a751-5405-4b37-8e71-114f956a6d09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:51.231466 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c5a751-5405-4b37-8e71-114f956a6d09 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:51.238598 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 98/208] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jun 15 14:30:50 2022] GET /identity/v3/role_assignments?user.id=aaa9a41e3d594efaaa8a9bd29ce1644e&scope.project.id=af60304e389b4295adfafbfc85459962&role.id=6ab04b221d1548c3b9c5941c17b9bd41 => generated 608 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:52.028732 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037d6ec9-4d55-4cc5-8745-6da67b198e5a None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:52.030352 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037d6ec9-4d55-4cc5-8745-6da67b198e5a None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:52.030854 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037d6ec9-4d55-4cc5-8745-6da67b198e5a None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:52.032591 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 111/209] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:30:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:52.038492 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67fec7f-0020-4053-8a13-a961dd24a5a4 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:52.039110 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67fec7f-0020-4053-8a13-a961dd24a5a4 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:52.039621 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67fec7f-0020-4053-8a13-a961dd24a5a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:52.059217 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-b67fec7f-0020-4053-8a13-a961dd24a5a4 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:52.060844 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-b67fec7f-0020-4053-8a13-a961dd24a5a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:52.061450 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bfd0a6-3373-476f-8d30-8ddcfc03cf09 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:52.061743 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bfd0a6-3373-476f-8d30-8ddcfc03cf09 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:52.062033 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bfd0a6-3373-476f-8d30-8ddcfc03cf09 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:52.063677 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 112/210] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:30:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:52.068530 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d6fd5-bccc-4754-933f-47966bc5da31 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:52.068821 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d6fd5-bccc-4754-933f-47966bc5da31 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:52.069106 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d6fd5-bccc-4754-933f-47966bc5da31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:52.090411 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-711d6fd5-bccc-4754-933f-47966bc5da31 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:52.091597 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-711d6fd5-bccc-4754-933f-47966bc5da31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:52.137340 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 99/211] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:30:52.160780 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67fec7f-0020-4053-8a13-a961dd24a5a4 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:52.172330 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 113/212] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:30:52.182329 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:52.197582 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711d6fd5-bccc-4754-933f-47966bc5da31 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:52.265827 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67fec7f-0020-4053-8a13-a961dd24a5a4 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:52.269030 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:52.302663 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096b11c9-41d2-4f56-84c5-d9cb82992ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:52.303743 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096b11c9-41d2-4f56-84c5-d9cb82992ad0 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:52.304219 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096b11c9-41d2-4f56-84c5-d9cb82992ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:52.304535 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711d6fd5-bccc-4754-933f-47966bc5da31 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:52.304792 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096b11c9-41d2-4f56-84c5-d9cb82992ad0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:52.316166 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096b11c9-41d2-4f56-84c5-d9cb82992ad0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:30:52.324200 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-096b11c9-41d2-4f56-84c5-d9cb82992ad0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:30:52.324200 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:52.324200 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:52.324200 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:52.324200 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:52.324200 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:52.324200 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:52.324200 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:52.324200 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:52.324200 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:52.324200 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:52.324200 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:52.324200 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:30:52.324200 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:30:52.324200 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:30:52.324200 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:30:52.325264 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:30:52.325264 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:30:52.325264 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:30:52.325264 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:30:52.325264 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:52.325264 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:52.325264 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:52.325264 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:52.325264 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:52.325264 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:52.325264 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:52.325264 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:52.325264 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:52.325264 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:52.325264 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:52.325264 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:52.326259 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:52.326259 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:52.326259 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:52.326259 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:52.326259 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:30:52.326259 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:30:52.326259 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:30:52.326259 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:30:52.326259 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:30:52.326259 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:30:52.326259 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:30:52.326259 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:30:52.333032 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 100/213] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jun 15 14:30:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:30:52.334184 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096b11c9-41d2-4f56-84c5-d9cb82992ad0 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:52.341501 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e83f70f-1c85-4900-844c-958c1ad43b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:52.342543 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e83f70f-1c85-4900-844c-958c1ad43b47 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:52.342970 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e83f70f-1c85-4900-844c-958c1ad43b47 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:52.343255 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e83f70f-1c85-4900-844c-958c1ad43b47 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:52.344467 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e83f70f-1c85-4900-844c-958c1ad43b47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:52.346048 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e83f70f-1c85-4900-844c-958c1ad43b47 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:52.352210 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 114/214] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:52.353495 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e83f70f-1c85-4900-844c-958c1ad43b47 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:52.436691 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:52.452444 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096b11c9-41d2-4f56-84c5-d9cb82992ad0 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:52.457015 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e83f70f-1c85-4900-844c-958c1ad43b47 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:52.501429 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae63dd7-649a-4deb-9e31-9eddfcd71606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:52.502225 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae63dd7-649a-4deb-9e31-9eddfcd71606 None admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:52.502582 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae63dd7-649a-4deb-9e31-9eddfcd71606 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:52.502936 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae63dd7-649a-4deb-9e31-9eddfcd71606 None admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:52.503731 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae63dd7-649a-4deb-9e31-9eddfcd71606 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:52.504261 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e472b315-3072-4e62-bfe3-7708993b2ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:52.505141 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472b315-3072-4e62-bfe3-7708993b2ed6 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:52.505435 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae63dd7-649a-4deb-9e31-9eddfcd71606 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:52.505777 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472b315-3072-4e62-bfe3-7708993b2ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:52.506223 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472b315-3072-4e62-bfe3-7708993b2ed6 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:52.507254 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e472b315-3072-4e62-bfe3-7708993b2ed6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:52.507747 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e472b315-3072-4e62-bfe3-7708993b2ed6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:52.509372 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e472b315-3072-4e62-bfe3-7708993b2ed6 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:52.514962 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:52.522143 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 101/215] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Jun 15 14:30:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:52.527762 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e472b315-3072-4e62-bfe3-7708993b2ed6 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:52.605219 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 115/216] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Jun 15 14:30:52 2022] POST /identity/v3/users => generated 297 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:30:52.652400 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e472b315-3072-4e62-bfe3-7708993b2ed6 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:52.694092 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2166445d-11c8-4370-947f-adcd2ecf99fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:52.695264 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2166445d-11c8-4370-947f-adcd2ecf99fc None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:52.695669 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2166445d-11c8-4370-947f-adcd2ecf99fc None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:52.698041 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2166445d-11c8-4370-947f-adcd2ecf99fc None admin] PATH_INFO: `/v3/users/a6a6e55f2033424997a39262b45cfd3b` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:52.707842 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:52.715353 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2166445d-11c8-4370-947f-adcd2ecf99fc None admin] RBAC: Authorizing `identity:get_user(user_id=a6a6e55f2033424997a39262b45cfd3b)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:52.717601 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2166445d-11c8-4370-947f-adcd2ecf99fc None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:52.720174 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 102/217] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jun 15 14:30:52 2022] GET /identity/v3/users/a6a6e55f2033424997a39262b45cfd3b => generated 297 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:52.726006 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae63dd7-649a-4deb-9e31-9eddfcd71606 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:52.842358 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae63dd7-649a-4deb-9e31-9eddfcd71606 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:52.844484 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3acd71e-8eca-423d-89ad-b4f9eb9f73c7 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:52.844866 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3acd71e-8eca-423d-89ad-b4f9eb9f73c7 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:52.845218 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3acd71e-8eca-423d-89ad-b4f9eb9f73c7 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:52.847524 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 103/218] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:30:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:30:52.853790 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de9add-f0c6-4fdd-a11b-2e5b004cf05f None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:52.854088 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de9add-f0c6-4fdd-a11b-2e5b004cf05f None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:52.854362 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de9add-f0c6-4fdd-a11b-2e5b004cf05f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:52.874143 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-69de9add-f0c6-4fdd-a11b-2e5b004cf05f None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:52.875342 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-69de9add-f0c6-4fdd-a11b-2e5b004cf05f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:52.886378 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c97a4a1-5b9b-453b-8a9f-828a714c6a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:52.887334 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c97a4a1-5b9b-453b-8a9f-828a714c6a1d None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:52.887823 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c97a4a1-5b9b-453b-8a9f-828a714c6a1d None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:52.888796 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c97a4a1-5b9b-453b-8a9f-828a714c6a1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:52.990334 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 104/219] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:30:53.017477 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de9add-f0c6-4fdd-a11b-2e5b004cf05f None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:53.017660 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:53.031983 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c97a4a1-5b9b-453b-8a9f-828a714c6a1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:53.033512 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c97a4a1-5b9b-453b-8a9f-828a714c6a1d None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:53.209068 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de9add-f0c6-4fdd-a11b-2e5b004cf05f None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:53.223653 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 116/220] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jun 15 14:30:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:30:53.229704 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c97a4a1-5b9b-453b-8a9f-828a714c6a1d None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:53.249863 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:53.265618 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e0bccc-b85e-4ec1-bc33-b1b101384ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:53.266425 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e0bccc-b85e-4ec1-bc33-b1b101384ea0 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:53.266982 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e0bccc-b85e-4ec1-bc33-b1b101384ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:53.266982 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e0bccc-b85e-4ec1-bc33-b1b101384ea0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:53.290382 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e0bccc-b85e-4ec1-bc33-b1b101384ea0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:30:53.300223 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-d1e0bccc-b85e-4ec1-bc33-b1b101384ea0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:30:53.300223 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:53.300223 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:53.300223 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:53.300223 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:53.300223 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:53.300223 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:53.300223 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:53.300223 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:53.300223 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:53.300223 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:53.300223 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:53.300223 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:30:53.300223 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:30:53.300223 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:30:53.300223 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:30:53.301180 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:30:53.301180 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:30:53.301180 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:30:53.301180 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:30:53.301180 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:53.301180 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:53.301180 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:53.301180 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:53.301180 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:53.301180 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:53.301180 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:53.301180 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:53.301180 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:53.301180 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:53.301180 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:53.301180 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:53.301965 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:53.301965 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:53.301965 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:53.301965 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:53.301965 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:30:53.301965 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:30:53.301965 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:30:53.301965 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:30:53.301965 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:30:53.301965 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:30:53.301965 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:30:53.301965 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:30:53.301965 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 105/221] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jun 15 14:30:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:53.303101 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f339c3d-740b-4978-a5f3-663c13bfdcf7 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:53.303334 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f339c3d-740b-4978-a5f3-663c13bfdcf7 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:53.303620 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f339c3d-740b-4978-a5f3-663c13bfdcf7 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:53.305620 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 106/222] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:30:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:53.309846 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f339c3d-740b-4978-a5f3-663c13bfdcf7 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:53.351692 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:53.363951 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c97a4a1-5b9b-453b-8a9f-828a714c6a1d None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:53.409454 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e7e3d7-bf20-41d9-83cd-b0942a1b90cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:53.410356 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e7e3d7-bf20-41d9-83cd-b0942a1b90cf None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:53.411168 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e7e3d7-bf20-41d9-83cd-b0942a1b90cf None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:53.411455 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e7e3d7-bf20-41d9-83cd-b0942a1b90cf None admin] PATH_INFO: `/v3/projects/e58ea743785943a9936197cc15d05a80` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:53.413097 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e7e3d7-bf20-41d9-83cd-b0942a1b90cf None admin] RBAC: Authorizing `identity:get_project(project_id=e58ea743785943a9936197cc15d05a80)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:53.415259 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e7e3d7-bf20-41d9-83cd-b0942a1b90cf None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:53.417598 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 117/223] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jun 15 14:30:53 2022] GET /identity/v3/projects/e58ea743785943a9936197cc15d05a80 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:53.421276 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d92de5-833d-40df-9fa8-a4b1b1c00f21 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:53.421833 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d92de5-833d-40df-9fa8-a4b1b1c00f21 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:53.422261 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d92de5-833d-40df-9fa8-a4b1b1c00f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:53.433745 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f339c3d-740b-4978-a5f3-663c13bfdcf7 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:53.447267 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-15d92de5-833d-40df-9fa8-a4b1b1c00f21 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:53.448826 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-15d92de5-833d-40df-9fa8-a4b1b1c00f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:53.473281 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecef5d1-026a-45dc-a5b3-39191022012a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:53.474176 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecef5d1-026a-45dc-a5b3-39191022012a None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:53.474524 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecef5d1-026a-45dc-a5b3-39191022012a None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:53.474806 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecef5d1-026a-45dc-a5b3-39191022012a None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:53.475567 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecef5d1-026a-45dc-a5b3-39191022012a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:53.475877 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecef5d1-026a-45dc-a5b3-39191022012a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:53.477832 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecef5d1-026a-45dc-a5b3-39191022012a None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:53.495147 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 107/224] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jun 15 14:30:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:53.496881 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecef5d1-026a-45dc-a5b3-39191022012a None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:53.539350 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 118/225] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:30:53.540436 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d92de5-833d-40df-9fa8-a4b1b1c00f21 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:53.566225 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:53.624155 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecef5d1-026a-45dc-a5b3-39191022012a None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:53.640593 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:53.650314 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d92de5-833d-40df-9fa8-a4b1b1c00f21 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:53.663926 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f54edf-3229-4832-9d0f-7a376df71d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:53.664955 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f54edf-3229-4832-9d0f-7a376df71d7b None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:53.665459 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f54edf-3229-4832-9d0f-7a376df71d7b None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:53.665946 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f54edf-3229-4832-9d0f-7a376df71d7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:53.666976 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f54edf-3229-4832-9d0f-7a376df71d7b None admin] RBAC: Adding query filter params (role.id=9effaf496b874d01bd3f6d0555ef2d69, scope.project.id=e58ea743785943a9936197cc15d05a80, user.id=a6a6e55f2033424997a39262b45cfd3b) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:53.667512 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f54edf-3229-4832-9d0f-7a376df71d7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:53.669180 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f54edf-3229-4832-9d0f-7a376df71d7b None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:53.684266 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 108/226] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jun 15 14:30:53 2022] GET /identity/v3/role_assignments?user.id=a6a6e55f2033424997a39262b45cfd3b&scope.project.id=e58ea743785943a9936197cc15d05a80&role.id=9effaf496b874d01bd3f6d0555ef2d69 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:53.686702 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f54edf-3229-4832-9d0f-7a376df71d7b None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:53.692939 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f911816-e165-4a88-a433-388139de1117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:53.693950 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f911816-e165-4a88-a433-388139de1117 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:53.694309 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f911816-e165-4a88-a433-388139de1117 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:53.694725 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f911816-e165-4a88-a433-388139de1117 None admin] PATH_INFO: `/v3/users/fe1e5bd379ef4b7ab3ea48ca4c9923ab` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:53.701035 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f911816-e165-4a88-a433-388139de1117 None admin] RBAC: Authorizing `identity:get_user(user_id=fe1e5bd379ef4b7ab3ea48ca4c9923ab)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:53.702830 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f911816-e165-4a88-a433-388139de1117 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:53.704699 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 119/227] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jun 15 14:30:53 2022] GET /identity/v3/users/fe1e5bd379ef4b7ab3ea48ca4c9923ab => generated 283 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:53.711805 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f911816-e165-4a88-a433-388139de1117 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:53.809277 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f54edf-3229-4832-9d0f-7a376df71d7b None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:53.826124 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:53.851242 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d6530c-e5b5-41e0-b2d1-40015e040a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:53.853898 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d6530c-e5b5-41e0-b2d1-40015e040a56 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:53.853898 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d6530c-e5b5-41e0-b2d1-40015e040a56 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:53.853898 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d6530c-e5b5-41e0-b2d1-40015e040a56 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:53.865927 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d6530c-e5b5-41e0-b2d1-40015e040a56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:30:53.883408 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f911816-e165-4a88-a433-388139de1117 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:53.884708 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-c6d6530c-e5b5-41e0-b2d1-40015e040a56 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:30:53.884708 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:53.884708 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:53.884708 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:53.884708 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:53.884708 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:53.884708 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:53.884708 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:53.884708 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:53.884708 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:53.884708 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:53.884708 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:53.884708 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:30:53.884708 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:30:53.884708 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:30:53.884708 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:30:53.885569 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:30:53.885569 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:30:53.885569 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:30:53.885569 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:30:53.885569 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:53.885569 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:53.885569 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:53.885569 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:53.885569 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:53.885569 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:53.885569 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:53.885569 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:53.885569 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:53.885569 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:53.885569 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:53.885569 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:53.886362 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:53.886362 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:53.886362 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:53.886362 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:53.886362 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:30:53.886362 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:30:53.886362 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:30:53.886362 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:30:53.886362 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:30:53.886362 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:30:53.886362 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:30:53.886362 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:30:53.889328 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 109/228] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Jun 15 14:30:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:53.894979 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d6530c-e5b5-41e0-b2d1-40015e040a56 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:53.970070 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367eecb2-bcb9-4abe-be09-5c3b8ca4ec73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:53.970953 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367eecb2-bcb9-4abe-be09-5c3b8ca4ec73 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:53.971237 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367eecb2-bcb9-4abe-be09-5c3b8ca4ec73 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:53.971512 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367eecb2-bcb9-4abe-be09-5c3b8ca4ec73 None admin] PATH_INFO: `/v3/projects/e58ea743785943a9936197cc15d05a80` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:53.975371 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367eecb2-bcb9-4abe-be09-5c3b8ca4ec73 None admin] RBAC: Authorizing `identity:get_project(project_id=e58ea743785943a9936197cc15d05a80)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:53.977009 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367eecb2-bcb9-4abe-be09-5c3b8ca4ec73 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:53.979733 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 120/229] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jun 15 14:30:53 2022] GET /identity/v3/projects/e58ea743785943a9936197cc15d05a80 => generated 302 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:53.997078 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367eecb2-bcb9-4abe-be09-5c3b8ca4ec73 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:54.014596 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d6530c-e5b5-41e0-b2d1-40015e040a56 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:54.046989 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:54.051734 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6647f5db-ed98-4ec8-ace8-b81f7a0e2144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:54.052669 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6647f5db-ed98-4ec8-ace8-b81f7a0e2144 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:54.053132 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6647f5db-ed98-4ec8-ace8-b81f7a0e2144 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:54.053643 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6647f5db-ed98-4ec8-ace8-b81f7a0e2144 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:54.054569 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6647f5db-ed98-4ec8-ace8-b81f7a0e2144 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:54.055051 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6647f5db-ed98-4ec8-ace8-b81f7a0e2144 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:54.056655 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6647f5db-ed98-4ec8-ace8-b81f7a0e2144 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:54.066039 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 110/230] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jun 15 14:30:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:54.071679 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6647f5db-ed98-4ec8-ace8-b81f7a0e2144 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:54.124542 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:54.183383 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367eecb2-bcb9-4abe-be09-5c3b8ca4ec73 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:54.192965 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6647f5db-ed98-4ec8-ace8-b81f7a0e2144 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:54.229027 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4172af3-62e4-47f9-842c-40fdc619329b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:54.230577 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4172af3-62e4-47f9-842c-40fdc619329b None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:54.231091 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4172af3-62e4-47f9-842c-40fdc619329b None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:54.231689 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4172af3-62e4-47f9-842c-40fdc619329b None admin] PATH_INFO: `/v3/users/fe1e5bd379ef4b7ab3ea48ca4c9923ab` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:54.233636 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4172af3-62e4-47f9-842c-40fdc619329b None admin] RBAC: Authorizing `identity:get_user(user_id=fe1e5bd379ef4b7ab3ea48ca4c9923ab)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:54.235321 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4172af3-62e4-47f9-842c-40fdc619329b None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:54.238248 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 111/231] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jun 15 14:30:54 2022] GET /identity/v3/users/fe1e5bd379ef4b7ab3ea48ca4c9923ab => generated 283 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:54.244990 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4172af3-62e4-47f9-842c-40fdc619329b None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:54.284311 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069d48be-f53b-41b7-8dbc-cbc0223ff669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:54.285213 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069d48be-f53b-41b7-8dbc-cbc0223ff669 None admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:54.285490 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069d48be-f53b-41b7-8dbc-cbc0223ff669 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:54.285917 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069d48be-f53b-41b7-8dbc-cbc0223ff669 None admin] PATH_INFO: `/v3/projects/e58ea743785943a9936197cc15d05a80/users/fe1e5bd379ef4b7ab3ea48ca4c9923ab/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:54.293426 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069d48be-f53b-41b7-8dbc-cbc0223ff669 None admin] RBAC: Authorizing `identity:create_grant(project_id=e58ea743785943a9936197cc15d05a80, user_id=fe1e5bd379ef4b7ab3ea48ca4c9923ab, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:54.295702 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069d48be-f53b-41b7-8dbc-cbc0223ff669 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:54.360201 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4172af3-62e4-47f9-842c-40fdc619329b None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:54.361713 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:54.391187 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 121/232] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Jun 15 14:30:53 2022] PUT /identity/v3/projects/e58ea743785943a9936197cc15d05a80/users/fe1e5bd379ef4b7ab3ea48ca4c9923ab/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:30:54.402306 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1c6e7d-4fab-491b-8446-fa0e9b4da19f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:54.403231 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1c6e7d-4fab-491b-8446-fa0e9b4da19f None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:54.403567 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1c6e7d-4fab-491b-8446-fa0e9b4da19f None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:54.403848 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1c6e7d-4fab-491b-8446-fa0e9b4da19f None admin] PATH_INFO: `/v3/projects/e58ea743785943a9936197cc15d05a80` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:54.405223 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1c6e7d-4fab-491b-8446-fa0e9b4da19f None admin] RBAC: Authorizing `identity:get_project(project_id=e58ea743785943a9936197cc15d05a80)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:54.406678 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1c6e7d-4fab-491b-8446-fa0e9b4da19f None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:54.408978 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 112/233] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jun 15 14:30:54 2022] GET /identity/v3/projects/e58ea743785943a9936197cc15d05a80 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:54.414781 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069d48be-f53b-41b7-8dbc-cbc0223ff669 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:54.555154 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:54.569178 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069d48be-f53b-41b7-8dbc-cbc0223ff669 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:54.616710 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d3d98c-2a71-4107-9a05-f914f834a136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:54.617983 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d3d98c-2a71-4107-9a05-f914f834a136 None admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:54.618503 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d3d98c-2a71-4107-9a05-f914f834a136 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:54.618987 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d3d98c-2a71-4107-9a05-f914f834a136 None admin] PATH_INFO: `/v3/projects/e58ea743785943a9936197cc15d05a80/users/fe1e5bd379ef4b7ab3ea48ca4c9923ab/roles/61e98f694ee741a9894b7e6db6dd11d3` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:54.626165 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d3d98c-2a71-4107-9a05-f914f834a136 None admin] RBAC: Authorizing `identity:create_grant(project_id=e58ea743785943a9936197cc15d05a80, user_id=fe1e5bd379ef4b7ab3ea48ca4c9923ab, role_id=61e98f694ee741a9894b7e6db6dd11d3)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:54.627953 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d3d98c-2a71-4107-9a05-f914f834a136 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:54.731060 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 122/234] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Jun 15 14:30:54 2022] PUT /identity/v3/projects/e58ea743785943a9936197cc15d05a80/users/fe1e5bd379ef4b7ab3ea48ca4c9923ab/roles/61e98f694ee741a9894b7e6db6dd11d3 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:30:55.347544 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e802bbd8-b85e-4f34-9925-1762f0d47946 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:55.348301 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e802bbd8-b85e-4f34-9925-1762f0d47946 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:55.348797 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e802bbd8-b85e-4f34-9925-1762f0d47946 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:55.350399 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 113/235] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:30:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:55.366039 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad63e0a-ce7b-4781-be4e-7f2e2bd82d86 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:55.367227 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad63e0a-ce7b-4781-be4e-7f2e2bd82d86 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:55.367715 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad63e0a-ce7b-4781-be4e-7f2e2bd82d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:55.389785 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-3ad63e0a-ce7b-4781-be4e-7f2e2bd82d86 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:55.391007 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-3ad63e0a-ce7b-4781-be4e-7f2e2bd82d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:55.450819 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e457d97-dffa-4900-b5ef-5de20f64c19d None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:55.451936 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e457d97-dffa-4900-b5ef-5de20f64c19d None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:55.452065 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e457d97-dffa-4900-b5ef-5de20f64c19d None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:55.453950 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 114/236] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:30:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:55.459375 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ad2af5-2d3f-4f32-bb57-b1a212fb1b05 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:55.459661 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ad2af5-2d3f-4f32-bb57-b1a212fb1b05 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:55.460251 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ad2af5-2d3f-4f32-bb57-b1a212fb1b05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:55.489916 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-45ad2af5-2d3f-4f32-bb57-b1a212fb1b05 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:55.491115 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-45ad2af5-2d3f-4f32-bb57-b1a212fb1b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:55.495158 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 123/237] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:30:55.519982 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad63e0a-ce7b-4781-be4e-7f2e2bd82d86 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:55.617285 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 115/238] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:30:55.644948 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ad2af5-2d3f-4f32-bb57-b1a212fb1b05 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:55.656064 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:55.700535 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad63e0a-ce7b-4781-be4e-7f2e2bd82d86 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:55.771722 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d82638-f483-44c2-9960-1982c66cfb49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:55.772561 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d82638-f483-44c2-9960-1982c66cfb49 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:55.772846 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d82638-f483-44c2-9960-1982c66cfb49 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:55.773123 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d82638-f483-44c2-9960-1982c66cfb49 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:55.786116 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d82638-f483-44c2-9960-1982c66cfb49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:30:55.793984 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-75d82638-f483-44c2-9960-1982c66cfb49 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:30:55.793984 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:55.793984 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:55.793984 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:55.793984 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:55.793984 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:55.793984 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:55.793984 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:55.793984 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:55.793984 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:55.793984 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:55.793984 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:55.793984 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:30:55.793984 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:30:55.793984 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:30:55.793984 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:30:55.795021 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:30:55.795021 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:30:55.795021 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:30:55.795021 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:30:55.795021 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:55.795021 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:55.795021 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:55.795021 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:55.795021 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:55.795021 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:55.795021 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:55.795021 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:55.795021 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:55.795021 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:55.795021 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:55.795021 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:55.796430 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:55.796430 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:55.796430 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:55.796430 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:55.796430 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:30:55.796430 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:30:55.796430 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:30:55.796430 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:30:55.796430 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:30:55.796430 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:30:55.796430 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:30:55.796430 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:30:55.800258 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 124/239] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jun 15 14:30:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:30:55.802850 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d82638-f483-44c2-9960-1982c66cfb49 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:55.840463 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:55.845909 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ad2af5-2d3f-4f32-bb57-b1a212fb1b05 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:55.915876 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94268875-b638-4825-83ec-767ae17e8dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:55.916734 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94268875-b638-4825-83ec-767ae17e8dd7 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:55.917016 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94268875-b638-4825-83ec-767ae17e8dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:55.917290 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94268875-b638-4825-83ec-767ae17e8dd7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:55.920104 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94268875-b638-4825-83ec-767ae17e8dd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:55.922187 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94268875-b638-4825-83ec-767ae17e8dd7 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:55.924272 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 116/240] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:55.932408 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94268875-b638-4825-83ec-767ae17e8dd7 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:55.958834 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d82638-f483-44c2-9960-1982c66cfb49 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:56.003772 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c9b181-faa0-4ef5-9673-adad20f784c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:56.004484 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c9b181-faa0-4ef5-9673-adad20f784c6 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:56.004794 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c9b181-faa0-4ef5-9673-adad20f784c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:56.005071 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c9b181-faa0-4ef5-9673-adad20f784c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:56.005908 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c9b181-faa0-4ef5-9673-adad20f784c6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:56.006222 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c9b181-faa0-4ef5-9673-adad20f784c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:56.009348 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c9b181-faa0-4ef5-9673-adad20f784c6 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:56.012408 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:56.023913 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 125/241] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Jun 15 14:30:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:56.030282 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c9b181-faa0-4ef5-9673-adad20f784c6 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:56.136703 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c9b181-faa0-4ef5-9673-adad20f784c6 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:56.152174 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94268875-b638-4825-83ec-767ae17e8dd7 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:56.173210 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7cb912-d34f-43da-b46d-8ab165fb84ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:56.174015 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7cb912-d34f-43da-b46d-8ab165fb84ea None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:56.174299 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7cb912-d34f-43da-b46d-8ab165fb84ea None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:56.174579 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7cb912-d34f-43da-b46d-8ab165fb84ea None admin] PATH_INFO: `/v3/users/c820666a125c4f51a1880bb1a2940cfa` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:56.189325 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7cb912-d34f-43da-b46d-8ab165fb84ea None admin] RBAC: Authorizing `identity:get_user(user_id=c820666a125c4f51a1880bb1a2940cfa)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:56.190800 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7cb912-d34f-43da-b46d-8ab165fb84ea None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:56.192541 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 126/242] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jun 15 14:30:56 2022] GET /identity/v3/users/c820666a125c4f51a1880bb1a2940cfa => generated 297 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:56.197998 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7cb912-d34f-43da-b46d-8ab165fb84ea None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:56.225880 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce0ad83-337f-4d5e-96dc-ae395ccf84c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:56.228072 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce0ad83-337f-4d5e-96dc-ae395ccf84c1 None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:56.228385 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce0ad83-337f-4d5e-96dc-ae395ccf84c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:56.228666 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce0ad83-337f-4d5e-96dc-ae395ccf84c1 None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:56.229392 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce0ad83-337f-4d5e-96dc-ae395ccf84c1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:56.234124 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce0ad83-337f-4d5e-96dc-ae395ccf84c1 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:56.300506 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:56.343323 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 117/243] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Jun 15 14:30:55 2022] POST /identity/v3/users => generated 293 bytes in 411 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:30:56.344662 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf570f18-549a-4b48-a636-1e22b05d0863 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:56.345138 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf570f18-549a-4b48-a636-1e22b05d0863 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:56.345242 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf570f18-549a-4b48-a636-1e22b05d0863 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:56.346629 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 118/244] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:30:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:56.356922 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7cb912-d34f-43da-b46d-8ab165fb84ea None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:56.358471 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c37ef39-6b96-4a38-870f-7e4da3e58f86 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:56.358652 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c37ef39-6b96-4a38-870f-7e4da3e58f86 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:56.358919 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c37ef39-6b96-4a38-870f-7e4da3e58f86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:56.406473 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f772ab9-d368-40b2-ac15-6ee0ffb97a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:56.407912 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f772ab9-d368-40b2-ac15-6ee0ffb97a75 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:56.408415 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f772ab9-d368-40b2-ac15-6ee0ffb97a75 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:56.408930 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f772ab9-d368-40b2-ac15-6ee0ffb97a75 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:56.417316 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-6c37ef39-6b96-4a38-870f-7e4da3e58f86 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:56.418571 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-6c37ef39-6b96-4a38-870f-7e4da3e58f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:56.471311 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:56.497850 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f772ab9-d368-40b2-ac15-6ee0ffb97a75 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:56.500552 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f772ab9-d368-40b2-ac15-6ee0ffb97a75 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:56.569174 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 119/245] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:30:56.570211 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:56.596399 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c37ef39-6b96-4a38-870f-7e4da3e58f86 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:56.631158 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 127/246] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jun 15 14:30:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 15 14:30:56.637213 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f772ab9-d368-40b2-ac15-6ee0ffb97a75 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:56.727726 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c37ef39-6b96-4a38-870f-7e4da3e58f86 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:56.774438 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be146bdd-5155-40b7-be9e-491c5b73d8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:56.774671 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f772ab9-d368-40b2-ac15-6ee0ffb97a75 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:56.775857 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be146bdd-5155-40b7-be9e-491c5b73d8dd None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:56.776325 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be146bdd-5155-40b7-be9e-491c5b73d8dd None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:56.777029 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be146bdd-5155-40b7-be9e-491c5b73d8dd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:56.792045 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be146bdd-5155-40b7-be9e-491c5b73d8dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:30:56.803260 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-be146bdd-5155-40b7-be9e-491c5b73d8dd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:30:56.803260 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:56.803260 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:56.803260 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:56.803260 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:56.803260 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:56.803260 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:56.803260 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:56.803260 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:56.803260 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:56.803260 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:56.803260 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:56.803260 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:30:56.803260 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:30:56.803260 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:30:56.803260 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:30:56.804438 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:30:56.804438 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:30:56.804438 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:30:56.804438 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:30:56.804438 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:56.804438 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:56.804438 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:56.804438 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:56.804438 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:56.804438 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:56.804438 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:56.804438 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:56.804438 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:56.804438 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:56.804438 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:56.804438 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:56.805352 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:56.805352 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:56.805352 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:56.805352 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:56.805352 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:30:56.805352 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:30:56.805352 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:30:56.805352 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:30:56.805352 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:30:56.805352 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:30:56.805352 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:30:56.805352 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:30:56.807785 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 120/247] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jun 15 14:30:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:56.815106 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be146bdd-5155-40b7-be9e-491c5b73d8dd None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:56.824331 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321028b-2e22-46ab-be62-8d44fa684802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:56.825289 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321028b-2e22-46ab-be62-8d44fa684802 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:56.825644 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321028b-2e22-46ab-be62-8d44fa684802 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:56.826064 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321028b-2e22-46ab-be62-8d44fa684802 None admin] PATH_INFO: `/v3/projects/e58ea743785943a9936197cc15d05a80` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:56.827520 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4321028b-2e22-46ab-be62-8d44fa684802 None admin] RBAC: Authorizing `identity:get_project(project_id=e58ea743785943a9936197cc15d05a80)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:56.831062 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4321028b-2e22-46ab-be62-8d44fa684802 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:56.833526 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 128/248] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jun 15 14:30:56 2022] GET /identity/v3/projects/e58ea743785943a9936197cc15d05a80 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:56.841305 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321028b-2e22-46ab-be62-8d44fa684802 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:56.841449 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:56.880373 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:56.965917 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be146bdd-5155-40b7-be9e-491c5b73d8dd None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:56.978996 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321028b-2e22-46ab-be62-8d44fa684802 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:57.147307 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5835d28-d2db-4d1d-ac0a-2cc13ed5aec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:57.148077 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5835d28-d2db-4d1d-ac0a-2cc13ed5aec1 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:57.148387 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5835d28-d2db-4d1d-ac0a-2cc13ed5aec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:57.148717 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5835d28-d2db-4d1d-ac0a-2cc13ed5aec1 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:57.149505 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5835d28-d2db-4d1d-ac0a-2cc13ed5aec1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:57.150061 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5835d28-d2db-4d1d-ac0a-2cc13ed5aec1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:57.151525 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5835d28-d2db-4d1d-ac0a-2cc13ed5aec1 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:57.160465 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ec58da-68a7-4859-89c5-26238dd9a5de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:57.161150 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec58da-68a7-4859-89c5-26238dd9a5de None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:57.161436 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec58da-68a7-4859-89c5-26238dd9a5de None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:57.161717 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec58da-68a7-4859-89c5-26238dd9a5de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:57.162568 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ec58da-68a7-4859-89c5-26238dd9a5de None admin] RBAC: Adding query filter params (role.id=6ab04b221d1548c3b9c5941c17b9bd41, scope.project.id=e58ea743785943a9936197cc15d05a80, user.id=c820666a125c4f51a1880bb1a2940cfa) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:57.162884 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ec58da-68a7-4859-89c5-26238dd9a5de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:57.164423 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ec58da-68a7-4859-89c5-26238dd9a5de None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:57.176310 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 121/249] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Jun 15 14:30:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:57.176521 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 129/250] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jun 15 14:30:56 2022] GET /identity/v3/role_assignments?user.id=c820666a125c4f51a1880bb1a2940cfa&scope.project.id=e58ea743785943a9936197cc15d05a80&role.id=6ab04b221d1548c3b9c5941c17b9bd41 => generated 259 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:57.181492 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5835d28-d2db-4d1d-ac0a-2cc13ed5aec1 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:57.220565 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d0cc64-4764-4970-93a6-eb83cac3b94e None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:57.220909 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d0cc64-4764-4970-93a6-eb83cac3b94e None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:57.221128 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d0cc64-4764-4970-93a6-eb83cac3b94e None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:57.223053 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 130/251] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:30:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:57.229712 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5738cef5-5dbd-4ba0-9148-e88ff8d92eb2 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:57.230024 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5738cef5-5dbd-4ba0-9148-e88ff8d92eb2 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:57.230254 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5738cef5-5dbd-4ba0-9148-e88ff8d92eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:57.251656 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-5738cef5-5dbd-4ba0-9148-e88ff8d92eb2 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:57.252909 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-5738cef5-5dbd-4ba0-9148-e88ff8d92eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:57.332436 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:57.390421 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 131/252] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:30:57.409340 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5835d28-d2db-4d1d-ac0a-2cc13ed5aec1 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:57.446019 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5738cef5-5dbd-4ba0-9148-e88ff8d92eb2 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:57.472451 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:57.487056 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2d2aef-af29-4990-ab87-c0e59bd9a5cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:57.487056 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2d2aef-af29-4990-ab87-c0e59bd9a5cf None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:57.487056 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2d2aef-af29-4990-ab87-c0e59bd9a5cf None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:57.487056 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2d2aef-af29-4990-ab87-c0e59bd9a5cf None admin] PATH_INFO: `/v3/users/a6a6e55f2033424997a39262b45cfd3b` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:57.487727 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2d2aef-af29-4990-ab87-c0e59bd9a5cf None admin] RBAC: Authorizing `identity:get_user(user_id=a6a6e55f2033424997a39262b45cfd3b)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:57.487727 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2d2aef-af29-4990-ab87-c0e59bd9a5cf None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:57.488213 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 122/253] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jun 15 14:30:57 2022] GET /identity/v3/users/a6a6e55f2033424997a39262b45cfd3b => generated 297 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:57.489443 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc8004b-852c-491b-910e-b0398fdc98f9 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:57.489796 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc8004b-852c-491b-910e-b0398fdc98f9 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:57.490079 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc8004b-852c-491b-910e-b0398fdc98f9 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:57.491314 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 123/254] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:30:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:57.496412 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb42c945-7e45-4f7d-a21a-4c0e6b0725a7 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:57.496698 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb42c945-7e45-4f7d-a21a-4c0e6b0725a7 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:57.496979 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb42c945-7e45-4f7d-a21a-4c0e6b0725a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:57.534985 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-eb42c945-7e45-4f7d-a21a-4c0e6b0725a7 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:57.534985 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-eb42c945-7e45-4f7d-a21a-4c0e6b0725a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:57.625191 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5738cef5-5dbd-4ba0-9148-e88ff8d92eb2 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:57.645327 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 124/255] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:30:57.646239 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb42c945-7e45-4f7d-a21a-4c0e6b0725a7 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:57.693815 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f252c22-d6ac-49e6-a875-22f8c56a7ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:57.694670 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f252c22-d6ac-49e6-a875-22f8c56a7ae5 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:57.694957 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f252c22-d6ac-49e6-a875-22f8c56a7ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:57.695232 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f252c22-d6ac-49e6-a875-22f8c56a7ae5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:57.712205 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f252c22-d6ac-49e6-a875-22f8c56a7ae5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:30:57.720221 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-2f252c22-d6ac-49e6-a875-22f8c56a7ae5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:30:57.720221 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:57.720221 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:57.720221 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:57.720221 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:57.720221 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:57.720221 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:57.720221 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:57.720221 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:57.720221 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:57.720221 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:57.720221 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:57.720221 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:30:57.720221 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:30:57.720221 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:30:57.720221 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:30:57.721126 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:30:57.721126 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:30:57.721126 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:30:57.721126 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:30:57.721126 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:57.721126 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:57.721126 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:57.721126 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:57.721126 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:57.721126 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:57.721126 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:57.721126 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:57.721126 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:57.721126 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:57.721126 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:57.721126 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:57.722370 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:57.722370 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:57.722370 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:57.722370 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:57.722370 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:30:57.722370 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:30:57.722370 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:30:57.722370 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:30:57.722370 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:30:57.722370 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:30:57.722370 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:30:57.722370 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:30:57.722370 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 132/256] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jun 15 14:30:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:30:57.723030 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f252c22-d6ac-49e6-a875-22f8c56a7ae5 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:57.779929 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:57.828894 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb42c945-7e45-4f7d-a21a-4c0e6b0725a7 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:57.856810 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:57.885116 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bd3adf-ab40-46b2-a356-1220c63e9482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:57.886357 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd3adf-ab40-46b2-a356-1220c63e9482 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:57.886843 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd3adf-ab40-46b2-a356-1220c63e9482 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:57.887318 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd3adf-ab40-46b2-a356-1220c63e9482 None admin] PATH_INFO: `/v3/projects/e58ea743785943a9936197cc15d05a80` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:57.890031 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bd3adf-ab40-46b2-a356-1220c63e9482 None admin] RBAC: Authorizing `identity:get_project(project_id=e58ea743785943a9936197cc15d05a80)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:57.891717 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bd3adf-ab40-46b2-a356-1220c63e9482 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:57.894565 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 125/257] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jun 15 14:30:57 2022] GET /identity/v3/projects/e58ea743785943a9936197cc15d05a80 => generated 302 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:57.896288 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bd3adf-ab40-46b2-a356-1220c63e9482 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:57.898766 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f252c22-d6ac-49e6-a875-22f8c56a7ae5 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:57.964182 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ede6f3-ffe4-4353-910f-83adf501e0cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:57.965341 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ede6f3-ffe4-4353-910f-83adf501e0cb None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:57.965827 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ede6f3-ffe4-4353-910f-83adf501e0cb None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:57.966322 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ede6f3-ffe4-4353-910f-83adf501e0cb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:57.983376 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ede6f3-ffe4-4353-910f-83adf501e0cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:30:57.993235 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-c0ede6f3-ffe4-4353-910f-83adf501e0cb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:30:57.993235 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:57.993235 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:57.993235 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:57.993235 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:57.993235 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:57.993235 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:57.993235 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:57.993235 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:57.993235 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:57.993235 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:57.993235 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:57.993235 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:30:57.993235 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:30:57.993235 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:30:57.993235 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:30:57.994113 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:30:57.994113 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:30:57.994113 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:30:57.994113 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:30:57.994113 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:57.994113 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:57.994113 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:57.994113 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:57.994113 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:57.994113 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:57.994113 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:57.994113 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:57.994113 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:57.994113 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:57.994113 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:57.994113 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:57.995336 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:57.995336 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:57.995336 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:57.995336 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:57.995336 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:30:57.995336 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:30:57.995336 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:30:57.995336 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:30:57.995336 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:30:57.995336 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:30:57.995336 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:30:57.995336 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:30:57.995336 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 133/258] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Jun 15 14:30:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:57.995336 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ede6f3-ffe4-4353-910f-83adf501e0cb None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:58.086965 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bd3adf-ab40-46b2-a356-1220c63e9482 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:58.148939 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83663fc-4917-4a36-8e7e-ca682fa1c962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:58.149607 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83663fc-4917-4a36-8e7e-ca682fa1c962 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:58.150144 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83663fc-4917-4a36-8e7e-ca682fa1c962 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:58.150416 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83663fc-4917-4a36-8e7e-ca682fa1c962 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:58.151165 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83663fc-4917-4a36-8e7e-ca682fa1c962 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:58.151460 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83663fc-4917-4a36-8e7e-ca682fa1c962 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:58.152880 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83663fc-4917-4a36-8e7e-ca682fa1c962 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:58.170843 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 126/259] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jun 15 14:30:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:58.171759 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83663fc-4917-4a36-8e7e-ca682fa1c962 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:58.181430 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:58.183487 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ede6f3-ffe4-4353-910f-83adf501e0cb None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:58.250874 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf647eb-9339-4867-809a-06c1f6d31551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:58.251772 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf647eb-9339-4867-809a-06c1f6d31551 None admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:58.252050 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf647eb-9339-4867-809a-06c1f6d31551 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:58.252353 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf647eb-9339-4867-809a-06c1f6d31551 None admin] PATH_INFO: `/v3/projects/e58ea743785943a9936197cc15d05a80/users/a6a6e55f2033424997a39262b45cfd3b/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:58.256060 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf647eb-9339-4867-809a-06c1f6d31551 None admin] RBAC: Authorizing `identity:create_grant(project_id=e58ea743785943a9936197cc15d05a80, user_id=a6a6e55f2033424997a39262b45cfd3b, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:58.257527 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf647eb-9339-4867-809a-06c1f6d31551 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:58.318908 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83663fc-4917-4a36-8e7e-ca682fa1c962 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:58.339185 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:58.345188 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 134/260] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Jun 15 14:30:57 2022] PUT /identity/v3/projects/e58ea743785943a9936197cc15d05a80/users/a6a6e55f2033424997a39262b45cfd3b/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:30:58.346004 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf647eb-9339-4867-809a-06c1f6d31551 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:58.383327 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083ae054-0246-473e-b33a-e5b7377a0de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:58.385121 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ae054-0246-473e-b33a-e5b7377a0de9 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:58.385416 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ae054-0246-473e-b33a-e5b7377a0de9 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:58.385694 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ae054-0246-473e-b33a-e5b7377a0de9 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:58.387306 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083ae054-0246-473e-b33a-e5b7377a0de9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:58.387618 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083ae054-0246-473e-b33a-e5b7377a0de9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:58.391211 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083ae054-0246-473e-b33a-e5b7377a0de9 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:58.411655 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 127/261] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jun 15 14:30:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:58.416677 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083ae054-0246-473e-b33a-e5b7377a0de9 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:58.543287 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf647eb-9339-4867-809a-06c1f6d31551 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:58.556180 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:58.596889 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa38715-8e91-47db-af0e-198ad053a3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:58.597694 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa38715-8e91-47db-af0e-198ad053a3cf None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:58.597974 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa38715-8e91-47db-af0e-198ad053a3cf None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:58.598249 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa38715-8e91-47db-af0e-198ad053a3cf None admin] PATH_INFO: `/v3/users/fe1e5bd379ef4b7ab3ea48ca4c9923ab` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:58.600911 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa38715-8e91-47db-af0e-198ad053a3cf None admin] RBAC: Authorizing `identity:get_user(user_id=fe1e5bd379ef4b7ab3ea48ca4c9923ab)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:58.606204 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa38715-8e91-47db-af0e-198ad053a3cf None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:58.609285 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 135/262] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jun 15 14:30:58 2022] GET /identity/v3/users/fe1e5bd379ef4b7ab3ea48ca4c9923ab => generated 283 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:58.615576 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa38715-8e91-47db-af0e-198ad053a3cf None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:58.631506 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083ae054-0246-473e-b33a-e5b7377a0de9 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:58.707211 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2875746-7c52-4a6c-b805-3ac89d5bd422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:58.708119 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2875746-7c52-4a6c-b805-3ac89d5bd422 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:58.708395 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2875746-7c52-4a6c-b805-3ac89d5bd422 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:58.708670 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2875746-7c52-4a6c-b805-3ac89d5bd422 None admin] PATH_INFO: `/v3/users/fe1e5bd379ef4b7ab3ea48ca4c9923ab` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:58.711921 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2875746-7c52-4a6c-b805-3ac89d5bd422 None admin] RBAC: Authorizing `identity:get_user(user_id=fe1e5bd379ef4b7ab3ea48ca4c9923ab)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:58.715123 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2875746-7c52-4a6c-b805-3ac89d5bd422 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:58.719077 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 128/263] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jun 15 14:30:58 2022] GET /identity/v3/users/fe1e5bd379ef4b7ab3ea48ca4c9923ab => generated 283 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:58.724218 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2875746-7c52-4a6c-b805-3ac89d5bd422 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:58.747016 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:58.788353 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa38715-8e91-47db-af0e-198ad053a3cf None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:58.823234 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b785724-d5ea-4624-a780-d3d8569ab97d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:58.824229 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b785724-d5ea-4624-a780-d3d8569ab97d None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:58.824643 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b785724-d5ea-4624-a780-d3d8569ab97d None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:58.825083 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b785724-d5ea-4624-a780-d3d8569ab97d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:58.878648 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2875746-7c52-4a6c-b805-3ac89d5bd422 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:58.897705 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b785724-d5ea-4624-a780-d3d8569ab97d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:58.899389 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b785724-d5ea-4624-a780-d3d8569ab97d None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:58.915520 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04e7f1d-1092-4c0a-bb3c-74b4680bedce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:58.916621 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e7f1d-1092-4c0a-bb3c-74b4680bedce None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:58.917012 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e7f1d-1092-4c0a-bb3c-74b4680bedce None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:58.917395 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e7f1d-1092-4c0a-bb3c-74b4680bedce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:58.921570 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:59.000215 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04e7f1d-1092-4c0a-bb3c-74b4680bedce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:59.001957 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04e7f1d-1092-4c0a-bb3c-74b4680bedce None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:59.027095 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:59.111767 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 136/264] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jun 15 14:30:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:30:59.115823 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 129/265] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jun 15 14:30:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:30:59.121245 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04e7f1d-1092-4c0a-bb3c-74b4680bedce None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:59.121379 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b785724-d5ea-4624-a780-d3d8569ab97d None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:59.173848 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:59.251792 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04e7f1d-1092-4c0a-bb3c-74b4680bedce None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:59.291966 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35c619d-3286-4e35-b754-beef9fdae8c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:59.292856 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c619d-3286-4e35-b754-beef9fdae8c1 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:59.293308 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c619d-3286-4e35-b754-beef9fdae8c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:59.293594 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c619d-3286-4e35-b754-beef9fdae8c1 None admin] PATH_INFO: `/v3/projects/e58ea743785943a9936197cc15d05a80` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:59.295346 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35c619d-3286-4e35-b754-beef9fdae8c1 None admin] RBAC: Authorizing `identity:get_project(project_id=e58ea743785943a9936197cc15d05a80)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:59.297395 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35c619d-3286-4e35-b754-beef9fdae8c1 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:59.299497 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 130/266] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jun 15 14:30:59 2022] GET /identity/v3/projects/e58ea743785943a9936197cc15d05a80 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:59.305800 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35c619d-3286-4e35-b754-beef9fdae8c1 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:59.382244 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b785724-d5ea-4624-a780-d3d8569ab97d None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:59.428691 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35c619d-3286-4e35-b754-beef9fdae8c1 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:59.430501 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:59.453109 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b2a72d-ea96-4d15-bd56-d9d4405790e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:59.453993 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b2a72d-ea96-4d15-bd56-d9d4405790e0 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:59.454277 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b2a72d-ea96-4d15-bd56-d9d4405790e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:59.454556 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b2a72d-ea96-4d15-bd56-d9d4405790e0 None admin] PATH_INFO: `/v3/projects/e58ea743785943a9936197cc15d05a80` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:59.455814 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b2a72d-ea96-4d15-bd56-d9d4405790e0 None admin] RBAC: Authorizing `identity:get_project(project_id=e58ea743785943a9936197cc15d05a80)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:59.457301 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b2a72d-ea96-4d15-bd56-d9d4405790e0 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:59.459037 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 137/267] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jun 15 14:30:59 2022] GET /identity/v3/projects/e58ea743785943a9936197cc15d05a80 => generated 302 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:59.465533 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b2a72d-ea96-4d15-bd56-d9d4405790e0 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:59.467075 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3998d986-4dfb-467d-b4b4-8a2a6f1bbff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:59.467929 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3998d986-4dfb-467d-b4b4-8a2a6f1bbff2 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:59.468389 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3998d986-4dfb-467d-b4b4-8a2a6f1bbff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:59.468672 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3998d986-4dfb-467d-b4b4-8a2a6f1bbff2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:59.469515 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3998d986-4dfb-467d-b4b4-8a2a6f1bbff2 None admin] RBAC: Adding query filter params (role.id=61e98f694ee741a9894b7e6db6dd11d3, scope.project.id=e58ea743785943a9936197cc15d05a80, user.id=fe1e5bd379ef4b7ab3ea48ca4c9923ab) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:59.469831 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3998d986-4dfb-467d-b4b4-8a2a6f1bbff2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:59.471274 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3998d986-4dfb-467d-b4b4-8a2a6f1bbff2 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:59.479119 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 131/268] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jun 15 14:30:59 2022] GET /identity/v3/role_assignments?user.id=fe1e5bd379ef4b7ab3ea48ca4c9923ab&scope.project.id=e58ea743785943a9936197cc15d05a80&role.id=61e98f694ee741a9894b7e6db6dd11d3 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:59.480515 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72485f7a-bf3a-4f7f-ae01-3752a050b653 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:59.480810 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72485f7a-bf3a-4f7f-ae01-3752a050b653 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:59.481091 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72485f7a-bf3a-4f7f-ae01-3752a050b653 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:59.483842 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 132/269] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:30:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:59.489590 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193405f9-125c-477d-9124-b2d85da9f7a7 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:59.489889 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193405f9-125c-477d-9124-b2d85da9f7a7 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:59.490158 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193405f9-125c-477d-9124-b2d85da9f7a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:59.509391 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-193405f9-125c-477d-9124-b2d85da9f7a7 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:59.510622 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-193405f9-125c-477d-9124-b2d85da9f7a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:59.527365 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:59.592223 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 133/270] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:30:59.614070 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193405f9-125c-477d-9124-b2d85da9f7a7 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:59.676708 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b2a72d-ea96-4d15-bd56-d9d4405790e0 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:59.719425 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:59.726268 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193405f9-125c-477d-9124-b2d85da9f7a7 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:59.767650 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab06c62-3a73-4839-8d52-870d5fa5a6b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:59.768680 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e315366c-375e-4aeb-b63d-4c9adabe8d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:59.768978 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab06c62-3a73-4839-8d52-870d5fa5a6b9 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:59.769409 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab06c62-3a73-4839-8d52-870d5fa5a6b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:59.769659 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e315366c-375e-4aeb-b63d-4c9adabe8d1f None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:59.769988 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab06c62-3a73-4839-8d52-870d5fa5a6b9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:59.770098 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e315366c-375e-4aeb-b63d-4c9adabe8d1f None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:59.770462 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e315366c-375e-4aeb-b63d-4c9adabe8d1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:59.771429 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e315366c-375e-4aeb-b63d-4c9adabe8d1f None admin] RBAC: Adding query filter params (role.id=2bd1979f53bb4853a50dd5bee6aa0782, scope.project.id=e58ea743785943a9936197cc15d05a80, user.id=fe1e5bd379ef4b7ab3ea48ca4c9923ab) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:59.771846 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e315366c-375e-4aeb-b63d-4c9adabe8d1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:59.772127 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab06c62-3a73-4839-8d52-870d5fa5a6b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:59.773642 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e315366c-375e-4aeb-b63d-4c9adabe8d1f None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:59.773980 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab06c62-3a73-4839-8d52-870d5fa5a6b9 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:59.780202 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 134/271] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:59.783150 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab06c62-3a73-4839-8d52-870d5fa5a6b9 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:59.789593 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 138/272] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jun 15 14:30:59 2022] GET /identity/v3/role_assignments?user.id=fe1e5bd379ef4b7ab3ea48ca4c9923ab&scope.project.id=e58ea743785943a9936197cc15d05a80&role.id=2bd1979f53bb4853a50dd5bee6aa0782 => generated 608 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:59.825125 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a723a92-c51a-4763-a210-bd67293f64fd None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:59.825600 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a723a92-c51a-4763-a210-bd67293f64fd None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:59.826493 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a723a92-c51a-4763-a210-bd67293f64fd None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:59.828079 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 139/273] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:30:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:30:59.834957 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c92e60-655f-4123-9f4f-08fb7bdb6c31 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:59.834957 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c92e60-655f-4123-9f4f-08fb7bdb6c31 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:59.835445 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c92e60-655f-4123-9f4f-08fb7bdb6c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:59.871511 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-a2c92e60-655f-4123-9f4f-08fb7bdb6c31 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:59.873724 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-a2c92e60-655f-4123-9f4f-08fb7bdb6c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:59.899443 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab06c62-3a73-4839-8d52-870d5fa5a6b9 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:59.939486 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7806cf50-e016-426f-9c50-56bfc8a3734f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:59.940480 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806cf50-e016-426f-9c50-56bfc8a3734f None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:59.940912 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806cf50-e016-426f-9c50-56bfc8a3734f None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:59.941336 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806cf50-e016-426f-9c50-56bfc8a3734f None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:59.942210 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7806cf50-e016-426f-9c50-56bfc8a3734f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:59.943818 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7806cf50-e016-426f-9c50-56bfc8a3734f None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:59.999623 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 140/274] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:31:00.028905 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c92e60-655f-4123-9f4f-08fb7bdb6c31 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:00.063167 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 135/275] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Jun 15 14:30:59 2022] POST /identity/v3/users => generated 293 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:31:00.075048 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:00.162299 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c92e60-655f-4123-9f4f-08fb7bdb6c31 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:00.204915 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f64f997-175b-4bc7-a386-ef6fd40fc032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:00.205744 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f64f997-175b-4bc7-a386-ef6fd40fc032 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:00.206025 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f64f997-175b-4bc7-a386-ef6fd40fc032 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:00.206300 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f64f997-175b-4bc7-a386-ef6fd40fc032 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:00.218835 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f64f997-175b-4bc7-a386-ef6fd40fc032 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:31:00.226643 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-6f64f997-175b-4bc7-a386-ef6fd40fc032 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:31:00.226643 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:00.226643 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:00.226643 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:00.226643 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:00.226643 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:00.226643 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:00.226643 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:00.226643 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:00.226643 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:00.226643 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:00.226643 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:00.226643 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:31:00.226643 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:31:00.226643 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:31:00.226643 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:31:00.227572 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:31:00.227572 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:31:00.227572 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:31:00.227572 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:31:00.227572 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:00.227572 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:00.227572 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:00.227572 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:00.227572 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:00.227572 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:00.227572 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:00.227572 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:00.227572 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:00.227572 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:00.227572 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:00.227572 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:00.228696 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:00.228696 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:00.228696 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:00.228696 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:00.228696 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:31:00.228696 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:31:00.228696 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:31:00.228696 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:31:00.228696 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:31:00.228696 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:31:00.228696 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:31:00.228696 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:00.228696 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 141/276] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jun 15 14:31:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:31:00.235751 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7806cf50-e016-426f-9c50-56bfc8a3734f None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:00.254941 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be5de6-bac6-4703-8ca9-fd7b69d483bd None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:00.255228 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be5de6-bac6-4703-8ca9-fd7b69d483bd None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:00.255486 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be5de6-bac6-4703-8ca9-fd7b69d483bd None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:00.257163 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 142/277] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:31:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:00.262665 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b35a4-0f65-4b2c-8221-a57dc368d943 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:00.262956 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b35a4-0f65-4b2c-8221-a57dc368d943 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:00.263241 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b35a4-0f65-4b2c-8221-a57dc368d943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:00.274419 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:00.283175 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-138b35a4-0f65-4b2c-8221-a57dc368d943 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:00.284481 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-138b35a4-0f65-4b2c-8221-a57dc368d943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:00.382451 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7806cf50-e016-426f-9c50-56bfc8a3734f None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:00.387428 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 143/278] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:31:00.411632 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138b35a4-0f65-4b2c-8221-a57dc368d943 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:00.425673 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4c65b0-c9d8-4299-8ad1-478658fa6d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:00.426348 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4c65b0-c9d8-4299-8ad1-478658fa6d0e None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:00.426630 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4c65b0-c9d8-4299-8ad1-478658fa6d0e None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:00.426905 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4c65b0-c9d8-4299-8ad1-478658fa6d0e None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:00.427655 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4c65b0-c9d8-4299-8ad1-478658fa6d0e None admin] RBAC: Adding query filter params (name=member) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:00.427966 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4c65b0-c9d8-4299-8ad1-478658fa6d0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:00.429367 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4c65b0-c9d8-4299-8ad1-478658fa6d0e None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:00.439685 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:00.442727 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 136/279] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Jun 15 14:31:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:00.448481 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4c65b0-c9d8-4299-8ad1-478658fa6d0e None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:00.541366 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138b35a4-0f65-4b2c-8221-a57dc368d943 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:00.554777 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:00.577756 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4c65b0-c9d8-4299-8ad1-478658fa6d0e None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:00.590539 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e945793-cd0d-47f6-81c4-0d21bdee9dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:00.591608 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e945793-cd0d-47f6-81c4-0d21bdee9dff None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:00.592069 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e945793-cd0d-47f6-81c4-0d21bdee9dff None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:00.592511 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e945793-cd0d-47f6-81c4-0d21bdee9dff None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:00.603958 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e945793-cd0d-47f6-81c4-0d21bdee9dff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:31:00.613290 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-1e945793-cd0d-47f6-81c4-0d21bdee9dff None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:31:00.613290 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:00.613290 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:00.613290 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:00.613290 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:00.613290 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:00.613290 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:00.613290 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:00.613290 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:00.613290 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:00.613290 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:00.613290 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:00.613290 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:31:00.613290 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:31:00.613290 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:31:00.613290 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:31:00.614218 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:31:00.614218 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:31:00.614218 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:31:00.614218 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:31:00.614218 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:00.614218 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:00.614218 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:00.614218 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:00.614218 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:00.614218 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:00.614218 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:00.614218 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:00.614218 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:00.614218 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:00.614218 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:00.614218 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:00.615001 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:00.615001 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:00.615001 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:00.615001 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:00.615001 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:31:00.615001 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:31:00.615001 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:31:00.615001 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:31:00.615001 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:31:00.615001 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:31:00.615001 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:31:00.615001 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:00.617980 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 144/280] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jun 15 14:31:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:31:00.625191 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e945793-cd0d-47f6-81c4-0d21bdee9dff None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:00.633720 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aec8027-0c38-44c9-8a4b-dc82bac7a781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:00.634723 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec8027-0c38-44c9-8a4b-dc82bac7a781 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:00.635006 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec8027-0c38-44c9-8a4b-dc82bac7a781 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:00.635280 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec8027-0c38-44c9-8a4b-dc82bac7a781 None admin] PATH_INFO: `/v3/users/de0e5278ed9745c5981ffdcb67255051` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:00.665857 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aec8027-0c38-44c9-8a4b-dc82bac7a781 None admin] RBAC: Authorizing `identity:get_user(user_id=de0e5278ed9745c5981ffdcb67255051)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:00.667476 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aec8027-0c38-44c9-8a4b-dc82bac7a781 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:00.669396 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 137/281] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jun 15 14:31:00 2022] GET /identity/v3/users/de0e5278ed9745c5981ffdcb67255051 => generated 293 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:31:00.688583 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aec8027-0c38-44c9-8a4b-dc82bac7a781 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:00.717352 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:00.737040 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e945793-cd0d-47f6-81c4-0d21bdee9dff None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:00.797504 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbffb700-9423-48a2-9c00-35b973225583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:00.798107 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbffb700-9423-48a2-9c00-35b973225583 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:00.798421 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbffb700-9423-48a2-9c00-35b973225583 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:00.798761 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbffb700-9423-48a2-9c00-35b973225583 None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:00.799524 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbffb700-9423-48a2-9c00-35b973225583 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:00.799836 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbffb700-9423-48a2-9c00-35b973225583 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:00.801261 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbffb700-9423-48a2-9c00-35b973225583 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:00.809216 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aec8027-0c38-44c9-8a4b-dc82bac7a781 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:00.820528 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 145/282] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Jun 15 14:31:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:00.826455 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbffb700-9423-48a2-9c00-35b973225583 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:00.875774 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf66e6c3-e645-47ba-835d-c001d1131557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:00.876515 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf66e6c3-e645-47ba-835d-c001d1131557 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:00.876802 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf66e6c3-e645-47ba-835d-c001d1131557 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:00.877078 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf66e6c3-e645-47ba-835d-c001d1131557 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:00.878280 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf66e6c3-e645-47ba-835d-c001d1131557 None admin] RBAC: Adding query filter params (role.id=9effaf496b874d01bd3f6d0555ef2d69, user.id=de0e5278ed9745c5981ffdcb67255051, scope.system=all) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:00.878599 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf66e6c3-e645-47ba-835d-c001d1131557 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:00.880378 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf66e6c3-e645-47ba-835d-c001d1131557 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:00.888519 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 138/283] 10.222.0.42 () {56 vars in 1308 bytes} [Wed Jun 15 14:31:00 2022] GET /identity/v3/role_assignments?user.id=de0e5278ed9745c5981ffdcb67255051&scope.system=all&role.id=9effaf496b874d01bd3f6d0555ef2d69 => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:00.971274 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbffb700-9423-48a2-9c00-35b973225583 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:01.009448 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12e8175-8bee-4865-8adb-1ceec36ad5c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:01.010343 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12e8175-8bee-4865-8adb-1ceec36ad5c0 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:01.010681 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12e8175-8bee-4865-8adb-1ceec36ad5c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:01.011033 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12e8175-8bee-4865-8adb-1ceec36ad5c0 None admin] PATH_INFO: `/v3/users/c820666a125c4f51a1880bb1a2940cfa` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:01.012333 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12e8175-8bee-4865-8adb-1ceec36ad5c0 None admin] RBAC: Authorizing `identity:get_user(user_id=c820666a125c4f51a1880bb1a2940cfa)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:01.014581 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12e8175-8bee-4865-8adb-1ceec36ad5c0 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:01.016786 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 146/284] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jun 15 14:31:00 2022] GET /identity/v3/users/c820666a125c4f51a1880bb1a2940cfa => generated 297 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:01.022320 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf66e6c3-e645-47ba-835d-c001d1131557 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:01.026509 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:01.145003 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf66e6c3-e645-47ba-835d-c001d1131557 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:01.200194 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba74a0f9-dc2b-4729-bd8a-0900987f2898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:01.201682 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba74a0f9-dc2b-4729-bd8a-0900987f2898 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:01.202185 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba74a0f9-dc2b-4729-bd8a-0900987f2898 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:01.202603 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba74a0f9-dc2b-4729-bd8a-0900987f2898 None admin] PATH_INFO: `/v3/projects/e58ea743785943a9936197cc15d05a80` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:01.203829 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba74a0f9-dc2b-4729-bd8a-0900987f2898 None admin] RBAC: Authorizing `identity:get_project(project_id=e58ea743785943a9936197cc15d05a80)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:01.205325 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba74a0f9-dc2b-4729-bd8a-0900987f2898 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:01.207128 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 139/285] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jun 15 14:31:01 2022] GET /identity/v3/projects/e58ea743785943a9936197cc15d05a80 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:01.212939 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12e8175-8bee-4865-8adb-1ceec36ad5c0 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:01.225135 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:01.321092 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12e8175-8bee-4865-8adb-1ceec36ad5c0 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:01.360462 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba28aacf-d9c3-4570-add2-024192ffb148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:01.361414 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba28aacf-d9c3-4570-add2-024192ffb148 None admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:01.361695 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba28aacf-d9c3-4570-add2-024192ffb148 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:01.361975 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba28aacf-d9c3-4570-add2-024192ffb148 None admin] PATH_INFO: `/v3/projects/e58ea743785943a9936197cc15d05a80/users/c820666a125c4f51a1880bb1a2940cfa/roles/6ab04b221d1548c3b9c5941c17b9bd41` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:01.364014 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba28aacf-d9c3-4570-add2-024192ffb148 None admin] RBAC: Authorizing `identity:create_grant(project_id=e58ea743785943a9936197cc15d05a80, user_id=c820666a125c4f51a1880bb1a2940cfa, role_id=6ab04b221d1548c3b9c5941c17b9bd41)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:01.365560 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba28aacf-d9c3-4570-add2-024192ffb148 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:01.445981 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 147/286] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Jun 15 14:31:01 2022] PUT /identity/v3/projects/e58ea743785943a9936197cc15d05a80/users/c820666a125c4f51a1880bb1a2940cfa/roles/6ab04b221d1548c3b9c5941c17b9bd41 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:31:01.653835 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee85456-5552-4fd9-8aa1-375cda3125b3 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:01.654458 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee85456-5552-4fd9-8aa1-375cda3125b3 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:01.654458 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee85456-5552-4fd9-8aa1-375cda3125b3 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:01.656048 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 140/287] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:31:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:01.666002 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a79fb6-57a9-494e-a756-50ca52d3fdf7 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:01.666345 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a79fb6-57a9-494e-a756-50ca52d3fdf7 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:01.666632 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a79fb6-57a9-494e-a756-50ca52d3fdf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:01.689277 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-08a79fb6-57a9-494e-a756-50ca52d3fdf7 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:01.690544 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-08a79fb6-57a9-494e-a756-50ca52d3fdf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:01.759824 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:01.770105 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 148/288] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:31:01.810150 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee85456-5552-4fd9-8aa1-375cda3125b3 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:01.902456 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:01.934915 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee85456-5552-4fd9-8aa1-375cda3125b3 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:01.970090 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fb1df9-8799-401f-a82e-9aceaf5ecca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:01.970886 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb1df9-8799-401f-a82e-9aceaf5ecca9 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:01.971261 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb1df9-8799-401f-a82e-9aceaf5ecca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:01.971542 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb1df9-8799-401f-a82e-9aceaf5ecca9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:01.983169 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fb1df9-8799-401f-a82e-9aceaf5ecca9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:31:01.990997 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-90fb1df9-8799-401f-a82e-9aceaf5ecca9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:31:01.990997 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:01.990997 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:01.990997 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:01.990997 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:01.990997 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:01.990997 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:01.990997 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:01.990997 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:01.990997 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:01.990997 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:01.990997 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:01.990997 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:31:01.990997 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:31:01.990997 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:31:01.990997 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:31:01.992003 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:31:01.992003 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:31:01.992003 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:31:01.992003 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:31:01.992003 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:01.992003 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:01.992003 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:01.992003 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:01.992003 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:01.992003 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:01.992003 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:01.992003 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:01.992003 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:01.992003 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:01.992003 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:01.992003 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:01.993234 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:01.993234 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:01.993234 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:01.993234 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:01.993234 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:31:01.993234 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:31:01.993234 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:31:01.993234 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:31:01.993234 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:31:01.993234 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:31:01.993234 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:31:01.993234 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:01.993234 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 141/289] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jun 15 14:31:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:01.998869 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a79fb6-57a9-494e-a756-50ca52d3fdf7 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:02.124206 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a79fb6-57a9-494e-a756-50ca52d3fdf7 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:02.166607 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527ed9a1-7f82-4bab-ab6a-9fa6469428b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:02.167287 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527ed9a1-7f82-4bab-ab6a-9fa6469428b8 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:02.167578 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527ed9a1-7f82-4bab-ab6a-9fa6469428b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:02.167893 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527ed9a1-7f82-4bab-ab6a-9fa6469428b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:02.168638 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527ed9a1-7f82-4bab-ab6a-9fa6469428b8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:02.168947 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527ed9a1-7f82-4bab-ab6a-9fa6469428b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:02.170906 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527ed9a1-7f82-4bab-ab6a-9fa6469428b8 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:02.179724 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 149/290] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Jun 15 14:31:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:02.184886 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fb1df9-8799-401f-a82e-9aceaf5ecca9 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:02.310989 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fb1df9-8799-401f-a82e-9aceaf5ecca9 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:02.339594 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:02.344639 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbe3d50-75e0-48a2-b190-cb05595c211d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:02.345568 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbe3d50-75e0-48a2-b190-cb05595c211d None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:02.345845 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbe3d50-75e0-48a2-b190-cb05595c211d None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:02.346116 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbe3d50-75e0-48a2-b190-cb05595c211d None admin] PATH_INFO: `/v3/users/a6a6e55f2033424997a39262b45cfd3b` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:02.347365 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbe3d50-75e0-48a2-b190-cb05595c211d None admin] RBAC: Authorizing `identity:get_user(user_id=a6a6e55f2033424997a39262b45cfd3b)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:02.348843 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbe3d50-75e0-48a2-b190-cb05595c211d None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:02.350681 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 142/291] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jun 15 14:31:02 2022] GET /identity/v3/users/a6a6e55f2033424997a39262b45cfd3b => generated 297 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:02.363685 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527ed9a1-7f82-4bab-ab6a-9fa6469428b8 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:02.469272 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:02.535947 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527ed9a1-7f82-4bab-ab6a-9fa6469428b8 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:02.586272 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a54a61-834a-4a94-9f4e-11fac310e166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:02.586941 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a54a61-834a-4a94-9f4e-11fac310e166 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:02.587225 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a54a61-834a-4a94-9f4e-11fac310e166 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:02.587505 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a54a61-834a-4a94-9f4e-11fac310e166 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:02.688238 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a54a61-834a-4a94-9f4e-11fac310e166 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:02.690045 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a54a61-834a-4a94-9f4e-11fac310e166 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:02.770850 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:02.828994 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 150/292] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jun 15 14:31:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:31:02.834921 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbe3d50-75e0-48a2-b190-cb05595c211d None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:02.970460 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbe3d50-75e0-48a2-b190-cb05595c211d None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:03.018515 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb8ec3-fd4f-4def-8fc1-b394a4cfca1a None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:03.019143 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb8ec3-fd4f-4def-8fc1-b394a4cfca1a None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:03.019655 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb8ec3-fd4f-4def-8fc1-b394a4cfca1a None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:03.021616 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 151/293] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:31:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:03.027150 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf974f2-422b-4e23-b6df-765a6d04f021 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:03.027668 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf974f2-422b-4e23-b6df-765a6d04f021 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:03.028183 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf974f2-422b-4e23-b6df-765a6d04f021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:03.069394 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-eaf974f2-422b-4e23-b6df-765a6d04f021 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:03.071262 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-eaf974f2-422b-4e23-b6df-765a6d04f021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:03.096439 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026a15d4-bf97-43eb-afb4-2fe4d41c867d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:03.097704 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026a15d4-bf97-43eb-afb4-2fe4d41c867d None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:03.099351 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026a15d4-bf97-43eb-afb4-2fe4d41c867d None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:03.099854 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026a15d4-bf97-43eb-afb4-2fe4d41c867d None admin] PATH_INFO: `/v3/projects/e58ea743785943a9936197cc15d05a80` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:03.102371 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026a15d4-bf97-43eb-afb4-2fe4d41c867d None admin] RBAC: Authorizing `identity:get_project(project_id=e58ea743785943a9936197cc15d05a80)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:03.105366 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026a15d4-bf97-43eb-afb4-2fe4d41c867d None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:03.112612 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 143/294] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jun 15 14:31:02 2022] GET /identity/v3/projects/e58ea743785943a9936197cc15d05a80 => generated 302 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:03.114379 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010af514-e413-46da-b699-f6e24facb7ed None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:03.114909 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010af514-e413-46da-b699-f6e24facb7ed None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:03.115428 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010af514-e413-46da-b699-f6e24facb7ed None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:03.116990 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 144/295] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:31:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:03.118096 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010af514-e413-46da-b699-f6e24facb7ed None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:03.180118 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 152/296] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:31:03.180118 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9447a81b-668b-4f4b-89ad-129cf2565dd4 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:03.180118 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9447a81b-668b-4f4b-89ad-129cf2565dd4 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:03.180118 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9447a81b-668b-4f4b-89ad-129cf2565dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:03.193847 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-9447a81b-668b-4f4b-89ad-129cf2565dd4 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:03.195312 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-9447a81b-668b-4f4b-89ad-129cf2565dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:03.228372 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:03.270568 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010af514-e413-46da-b699-f6e24facb7ed None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:03.276913 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 153/297] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:31:03.277684 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9447a81b-668b-4f4b-89ad-129cf2565dd4 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:03.315768 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:03.328262 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee949113-a956-4748-8434-1312ff219161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:03.329397 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee949113-a956-4748-8434-1312ff219161 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:03.330662 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee949113-a956-4748-8434-1312ff219161 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:03.331155 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee949113-a956-4748-8434-1312ff219161 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:03.332240 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee949113-a956-4748-8434-1312ff219161 None admin] RBAC: Adding query filter params (role.id=9effaf496b874d01bd3f6d0555ef2d69, scope.project.id=e58ea743785943a9936197cc15d05a80, user.id=a6a6e55f2033424997a39262b45cfd3b) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:03.332763 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee949113-a956-4748-8434-1312ff219161 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:03.334505 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee949113-a956-4748-8434-1312ff219161 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:03.343205 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 145/298] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jun 15 14:31:03 2022] GET /identity/v3/role_assignments?user.id=a6a6e55f2033424997a39262b45cfd3b&scope.project.id=e58ea743785943a9936197cc15d05a80&role.id=9effaf496b874d01bd3f6d0555ef2d69 => generated 608 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:03.344836 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee949113-a956-4748-8434-1312ff219161 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:03.455454 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9447a81b-668b-4f4b-89ad-129cf2565dd4 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:03.496060 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3688b340-d949-4af5-9e9f-77c17c397eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:03.497295 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3688b340-d949-4af5-9e9f-77c17c397eac None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:03.498403 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3688b340-d949-4af5-9e9f-77c17c397eac None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:03.498889 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3688b340-d949-4af5-9e9f-77c17c397eac None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:03.511239 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3688b340-d949-4af5-9e9f-77c17c397eac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:31:03.520347 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-3688b340-d949-4af5-9e9f-77c17c397eac None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:31:03.520347 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:03.520347 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:03.520347 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:03.520347 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:03.520347 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:03.520347 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:03.520347 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:03.520347 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:03.520347 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:03.520347 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:03.520347 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:03.520347 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:31:03.520347 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:31:03.520347 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:31:03.520347 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:31:03.521305 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:31:03.521305 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:31:03.521305 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:31:03.521305 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:31:03.521305 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:03.521305 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:03.521305 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:03.521305 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:03.521305 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:03.521305 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:03.521305 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:03.521305 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:03.521305 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:03.521305 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:03.521305 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:03.521305 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:03.522107 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:03.522107 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:03.522107 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:03.522107 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:03.522107 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:31:03.522107 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:31:03.522107 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:31:03.522107 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:31:03.522107 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:31:03.522107 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:31:03.522107 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:31:03.522107 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:03.524623 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 154/299] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jun 15 14:31:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:03.531072 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3688b340-d949-4af5-9e9f-77c17c397eac None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:03.587355 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee949113-a956-4748-8434-1312ff219161 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:03.628840 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:03.635806 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e0eeb4-6ce4-49aa-af01-bd1f057d4edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:03.637479 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e0eeb4-6ce4-49aa-af01-bd1f057d4edb None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:03.637975 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e0eeb4-6ce4-49aa-af01-bd1f057d4edb None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:03.639001 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e0eeb4-6ce4-49aa-af01-bd1f057d4edb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:03.640481 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e0eeb4-6ce4-49aa-af01-bd1f057d4edb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:03.642886 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e0eeb4-6ce4-49aa-af01-bd1f057d4edb None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:03.643879 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:03.646563 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 146/300] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:31:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:03.653260 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e0eeb4-6ce4-49aa-af01-bd1f057d4edb None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:03.668655 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3688b340-d949-4af5-9e9f-77c17c397eac None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:03.711087 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af6a1a4-cdd5-4cc0-9797-0f779443fbf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:03.712147 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6a1a4-cdd5-4cc0-9797-0f779443fbf5 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:03.712672 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6a1a4-cdd5-4cc0-9797-0f779443fbf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:03.713176 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6a1a4-cdd5-4cc0-9797-0f779443fbf5 None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:03.714277 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af6a1a4-cdd5-4cc0-9797-0f779443fbf5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:03.714857 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af6a1a4-cdd5-4cc0-9797-0f779443fbf5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:03.716570 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af6a1a4-cdd5-4cc0-9797-0f779443fbf5 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:03.733713 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 155/301] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Jun 15 14:31:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:31:03.735667 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af6a1a4-cdd5-4cc0-9797-0f779443fbf5 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:03.773808 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e0eeb4-6ce4-49aa-af01-bd1f057d4edb None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:03.809508 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f395247a-8d7d-4560-988d-f73ae848c50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:03.810102 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f395247a-8d7d-4560-988d-f73ae848c50e None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:03.811010 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f395247a-8d7d-4560-988d-f73ae848c50e None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:03.811286 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f395247a-8d7d-4560-988d-f73ae848c50e None admin] PATH_INFO: `/v3/groups` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:03.811978 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f395247a-8d7d-4560-988d-f73ae848c50e None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:03.813440 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f395247a-8d7d-4560-988d-f73ae848c50e None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:03.845488 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af6a1a4-cdd5-4cc0-9797-0f779443fbf5 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:03.879301 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191c772c-9266-4ffe-a148-920ba29207da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:03.880200 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191c772c-9266-4ffe-a148-920ba29207da None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:03.880483 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191c772c-9266-4ffe-a148-920ba29207da None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:03.880760 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191c772c-9266-4ffe-a148-920ba29207da None admin] PATH_INFO: `/v3/users/868ddf688ff64f0f81a56f92e9c3ccf9` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:03.890115 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 147/302] 10.222.0.42 () {60 vars in 1134 bytes} [Wed Jun 15 14:31:03 2022] POST /identity/v3/groups => generated 229 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:31:03.896768 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:03.896768 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191c772c-9266-4ffe-a148-920ba29207da None admin] RBAC: Authorizing `identity:get_user(user_id=868ddf688ff64f0f81a56f92e9c3ccf9)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:03.897937 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191c772c-9266-4ffe-a148-920ba29207da None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:03.899534 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5001fb99-ecc2-4c79-92d1-e89868478b21 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:03.899958 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 156/303] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jun 15 14:31:03 2022] GET /identity/v3/users/868ddf688ff64f0f81a56f92e9c3ccf9 => generated 293 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:31:03.905803 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5001fb99-ecc2-4c79-92d1-e89868478b21 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:03.905803 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5001fb99-ecc2-4c79-92d1-e89868478b21 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:03.905803 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 148/304] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:31:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:03.910166 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191c772c-9266-4ffe-a148-920ba29207da None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:03.910771 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ccbae7-27ff-48d1-9cd2-9693a7204dd0 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:03.911175 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ccbae7-27ff-48d1-9cd2-9693a7204dd0 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:03.911460 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ccbae7-27ff-48d1-9cd2-9693a7204dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:03.930425 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-36ccbae7-27ff-48d1-9cd2-9693a7204dd0 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:03.931632 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-36ccbae7-27ff-48d1-9cd2-9693a7204dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:04.012015 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:04.028299 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 149/305] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:31:04.039903 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191c772c-9266-4ffe-a148-920ba29207da None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:04.054230 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ccbae7-27ff-48d1-9cd2-9693a7204dd0 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:04.095895 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3543cc14-40a9-42c5-aeba-9f3081f5c0bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:04.095895 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3543cc14-40a9-42c5-aeba-9f3081f5c0bf None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:04.096182 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3543cc14-40a9-42c5-aeba-9f3081f5c0bf None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:04.096404 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3543cc14-40a9-42c5-aeba-9f3081f5c0bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:04.097316 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3543cc14-40a9-42c5-aeba-9f3081f5c0bf None admin] RBAC: Adding query filter params (role.id=6ab04b221d1548c3b9c5941c17b9bd41, user.id=868ddf688ff64f0f81a56f92e9c3ccf9, scope.system=all) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:04.097635 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3543cc14-40a9-42c5-aeba-9f3081f5c0bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:04.099922 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3543cc14-40a9-42c5-aeba-9f3081f5c0bf None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:04.105933 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 157/306] 10.222.0.42 () {56 vars in 1308 bytes} [Wed Jun 15 14:31:03 2022] GET /identity/v3/role_assignments?user.id=868ddf688ff64f0f81a56f92e9c3ccf9&scope.system=all&role.id=6ab04b221d1548c3b9c5941c17b9bd41 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:04.113928 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49b073e-9f06-4ee2-9c5b-1f1d95b770a3 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:04.114222 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49b073e-9f06-4ee2-9c5b-1f1d95b770a3 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:04.114826 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49b073e-9f06-4ee2-9c5b-1f1d95b770a3 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:04.115891 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 158/307] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:31:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:04.122182 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46447c94-c70e-4d9c-b0e1-afa492c34d50 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:04.122478 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46447c94-c70e-4d9c-b0e1-afa492c34d50 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:04.123328 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46447c94-c70e-4d9c-b0e1-afa492c34d50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:04.143230 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-46447c94-c70e-4d9c-b0e1-afa492c34d50 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:04.144508 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-46447c94-c70e-4d9c-b0e1-afa492c34d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:04.181323 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ccbae7-27ff-48d1-9cd2-9693a7204dd0 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:04.220848 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b226276-2269-46bc-8952-e09db1e0a633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:04.221666 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b226276-2269-46bc-8952-e09db1e0a633 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:04.221954 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b226276-2269-46bc-8952-e09db1e0a633 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:04.222234 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b226276-2269-46bc-8952-e09db1e0a633 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:04.235049 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b226276-2269-46bc-8952-e09db1e0a633 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:31:04.240474 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:04.243272 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-4b226276-2269-46bc-8952-e09db1e0a633 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:31:04.243272 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:04.243272 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:04.243272 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:04.243272 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:04.243272 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:04.243272 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:04.243272 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:04.243272 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:04.243272 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:04.243272 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:04.243272 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:04.243272 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:31:04.243272 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:31:04.243272 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:31:04.243272 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:31:04.244241 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:31:04.244241 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:31:04.244241 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:31:04.244241 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:31:04.244241 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:04.244241 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:04.244241 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:04.244241 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:04.244241 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:04.244241 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:04.244241 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:04.244241 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:04.244241 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:04.244241 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:04.244241 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:04.244241 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:04.245033 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:04.245033 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:04.245033 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:04.245033 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:04.245033 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:31:04.245033 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:31:04.245033 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:31:04.245033 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:31:04.245033 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:31:04.245033 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:31:04.245033 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:31:04.245033 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:04.245033 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 150/308] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jun 15 14:31:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:31:04.250758 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b226276-2269-46bc-8952-e09db1e0a633 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:04.259121 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 159/309] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:31:04.285770 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46447c94-c70e-4d9c-b0e1-afa492c34d50 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:04.317606 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:04.388047 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b226276-2269-46bc-8952-e09db1e0a633 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:04.402560 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46447c94-c70e-4d9c-b0e1-afa492c34d50 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:04.436606 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a15e8d1-74c7-496e-a382-79f03ec10f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:04.437554 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a15e8d1-74c7-496e-a382-79f03ec10f90 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:04.437840 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a15e8d1-74c7-496e-a382-79f03ec10f90 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:04.438127 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a15e8d1-74c7-496e-a382-79f03ec10f90 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:04.439054 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a15e8d1-74c7-496e-a382-79f03ec10f90 None admin] RBAC: Adding query filter params (name=member) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:04.439426 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a15e8d1-74c7-496e-a382-79f03ec10f90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:04.441237 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d9d067-5a84-419e-90e3-361c707ebbb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:04.442099 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d9d067-5a84-419e-90e3-361c707ebbb7 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:04.442433 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d9d067-5a84-419e-90e3-361c707ebbb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:04.442764 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d9d067-5a84-419e-90e3-361c707ebbb7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:04.445256 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a15e8d1-74c7-496e-a382-79f03ec10f90 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:04.454807 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 151/310] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Jun 15 14:31:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:31:04.454986 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d9d067-5a84-419e-90e3-361c707ebbb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:31:04.461628 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a15e8d1-74c7-496e-a382-79f03ec10f90 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:04.462630 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-16d9d067-5a84-419e-90e3-361c707ebbb7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:31:04.462630 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:04.462630 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:04.462630 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:04.462630 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:04.462630 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:04.462630 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:04.462630 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:04.462630 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:04.462630 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:04.462630 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:04.462630 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:04.462630 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:31:04.462630 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:31:04.462630 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:31:04.462630 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:31:04.463480 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:31:04.463480 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:31:04.463480 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:31:04.463480 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:31:04.463480 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:04.463480 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:04.463480 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:04.463480 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:04.463480 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:04.463480 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:04.463480 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:04.463480 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:04.463480 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:04.463480 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:04.463480 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:04.463480 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:04.464328 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:04.464328 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:04.464328 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:04.464328 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:04.464328 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:31:04.464328 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:31:04.464328 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:31:04.464328 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:31:04.464328 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:31:04.464328 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:31:04.464328 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:31:04.464328 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:04.468849 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 160/311] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jun 15 14:31:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:04.469711 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d9d067-5a84-419e-90e3-361c707ebbb7 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:04.536737 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:04.593707 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d9d067-5a84-419e-90e3-361c707ebbb7 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:04.596113 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a15e8d1-74c7-496e-a382-79f03ec10f90 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:04.624475 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:04.630466 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37194ebf-fc11-47ea-a632-6c34118cd288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:04.631132 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37194ebf-fc11-47ea-a632-6c34118cd288 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:04.631412 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37194ebf-fc11-47ea-a632-6c34118cd288 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:04.631686 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37194ebf-fc11-47ea-a632-6c34118cd288 None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:04.632448 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37194ebf-fc11-47ea-a632-6c34118cd288 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:04.632754 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37194ebf-fc11-47ea-a632-6c34118cd288 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:04.634122 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37194ebf-fc11-47ea-a632-6c34118cd288 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:04.635964 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f510bef-ddf3-4c96-9fdc-40535194b817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:04.637346 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f510bef-ddf3-4c96-9fdc-40535194b817 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:04.637676 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f510bef-ddf3-4c96-9fdc-40535194b817 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:04.637951 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f510bef-ddf3-4c96-9fdc-40535194b817 None admin] PATH_INFO: `/v3/users/de0e5278ed9745c5981ffdcb67255051` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:04.639915 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f510bef-ddf3-4c96-9fdc-40535194b817 None admin] RBAC: Authorizing `identity:get_user(user_id=de0e5278ed9745c5981ffdcb67255051)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:04.642234 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f510bef-ddf3-4c96-9fdc-40535194b817 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:04.648459 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 161/313] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Jun 15 14:31:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:31:04.648605 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 152/313] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jun 15 14:31:04 2022] GET /identity/v3/users/de0e5278ed9745c5981ffdcb67255051 => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:04.649329 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37194ebf-fc11-47ea-a632-6c34118cd288 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:04.650113 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f510bef-ddf3-4c96-9fdc-40535194b817 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:04.754658 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37194ebf-fc11-47ea-a632-6c34118cd288 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:04.773013 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:04.787915 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f510bef-ddf3-4c96-9fdc-40535194b817 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:04.797265 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb020bc-17dd-48ae-97c3-365e511c5e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:04.798241 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb020bc-17dd-48ae-97c3-365e511c5e79 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:04.798591 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb020bc-17dd-48ae-97c3-365e511c5e79 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:04.799327 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb020bc-17dd-48ae-97c3-365e511c5e79 None admin] PATH_INFO: `/v3/users/c820666a125c4f51a1880bb1a2940cfa` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:04.801011 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb020bc-17dd-48ae-97c3-365e511c5e79 None admin] RBAC: Authorizing `identity:get_user(user_id=c820666a125c4f51a1880bb1a2940cfa)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:04.802800 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb020bc-17dd-48ae-97c3-365e511c5e79 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:04.805284 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 162/314] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jun 15 14:31:04 2022] GET /identity/v3/users/c820666a125c4f51a1880bb1a2940cfa => generated 297 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:04.812118 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb020bc-17dd-48ae-97c3-365e511c5e79 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:04.828210 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9f72a3-af8d-4b6f-8710-506a514db87d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:04.829076 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f72a3-af8d-4b6f-8710-506a514db87d None admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:04.829360 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f72a3-af8d-4b6f-8710-506a514db87d None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:04.829638 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f72a3-af8d-4b6f-8710-506a514db87d None admin] PATH_INFO: `/v3/system/users/de0e5278ed9745c5981ffdcb67255051/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:04.831361 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9f72a3-af8d-4b6f-8710-506a514db87d None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=de0e5278ed9745c5981ffdcb67255051, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:04.832531 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9f72a3-af8d-4b6f-8710-506a514db87d None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:04.896714 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 153/315] 10.222.0.42 () {58 vars in 1334 bytes} [Wed Jun 15 14:31:04 2022] PUT /identity/v3/system/users/de0e5278ed9745c5981ffdcb67255051/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:31:04.936401 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb020bc-17dd-48ae-97c3-365e511c5e79 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:04.975343 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed72c3f-d2f5-4da4-b1ee-5e292a51c500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:04.976072 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed72c3f-d2f5-4da4-b1ee-5e292a51c500 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:04.976383 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed72c3f-d2f5-4da4-b1ee-5e292a51c500 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:04.976709 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed72c3f-d2f5-4da4-b1ee-5e292a51c500 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:05.036798 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:05.057419 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed72c3f-d2f5-4da4-b1ee-5e292a51c500 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:05.058938 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed72c3f-d2f5-4da4-b1ee-5e292a51c500 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:05.176378 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 163/316] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jun 15 14:31:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 15 14:31:05.182217 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9f72a3-af8d-4b6f-8710-506a514db87d None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:05.230262 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:05.295024 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9f72a3-af8d-4b6f-8710-506a514db87d None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:05.331227 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f5b766-8348-410c-b662-b4dd882edb5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:05.332462 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f5b766-8348-410c-b662-b4dd882edb5e None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:05.332890 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f5b766-8348-410c-b662-b4dd882edb5e None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:05.333313 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f5b766-8348-410c-b662-b4dd882edb5e None admin] PATH_INFO: `/v3/projects/e58ea743785943a9936197cc15d05a80` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:05.334650 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f5b766-8348-410c-b662-b4dd882edb5e None admin] RBAC: Authorizing `identity:get_project(project_id=e58ea743785943a9936197cc15d05a80)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:05.336272 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f5b766-8348-410c-b662-b4dd882edb5e None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:05.338405 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 154/317] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jun 15 14:31:05 2022] GET /identity/v3/projects/e58ea743785943a9936197cc15d05a80 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:05.344966 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed72c3f-d2f5-4da4-b1ee-5e292a51c500 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:05.435782 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:05.452066 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed72c3f-d2f5-4da4-b1ee-5e292a51c500 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:05.489504 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b643fa7-fe55-4976-898e-30fbdb81acd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:05.490161 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b643fa7-fe55-4976-898e-30fbdb81acd2 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:05.490441 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b643fa7-fe55-4976-898e-30fbdb81acd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:05.490719 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b643fa7-fe55-4976-898e-30fbdb81acd2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:05.491547 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b643fa7-fe55-4976-898e-30fbdb81acd2 None admin] RBAC: Adding query filter params (role.id=6ab04b221d1548c3b9c5941c17b9bd41, scope.project.id=e58ea743785943a9936197cc15d05a80, user.id=c820666a125c4f51a1880bb1a2940cfa) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:05.491859 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b643fa7-fe55-4976-898e-30fbdb81acd2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:05.493344 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b643fa7-fe55-4976-898e-30fbdb81acd2 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:05.499251 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 164/318] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jun 15 14:31:05 2022] GET /identity/v3/role_assignments?user.id=c820666a125c4f51a1880bb1a2940cfa&scope.project.id=e58ea743785943a9936197cc15d05a80&role.id=6ab04b221d1548c3b9c5941c17b9bd41 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:06.219415 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d2c3a-c662-4284-a865-9c6b79c316a9 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:06.219932 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d2c3a-c662-4284-a865-9c6b79c316a9 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:06.220036 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d2c3a-c662-4284-a865-9c6b79c316a9 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:06.221428 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 155/319] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:31:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:06.226748 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeffc09c-b0a5-4221-b99c-097b965caa8a None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:06.227229 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeffc09c-b0a5-4221-b99c-097b965caa8a None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:06.227301 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeffc09c-b0a5-4221-b99c-097b965caa8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:06.246638 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-aeffc09c-b0a5-4221-b99c-097b965caa8a None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:06.247901 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-aeffc09c-b0a5-4221-b99c-097b965caa8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:06.300941 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da739a8-891f-4476-91b9-3f859f1b32a3 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:06.301264 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da739a8-891f-4476-91b9-3f859f1b32a3 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:06.301503 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da739a8-891f-4476-91b9-3f859f1b32a3 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:06.303012 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 156/320] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:31:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:31:06.307918 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd9defe-d3bc-44f5-ac56-8ee639082c16 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:06.308255 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd9defe-d3bc-44f5-ac56-8ee639082c16 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:06.308538 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd9defe-d3bc-44f5-ac56-8ee639082c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:06.319995 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 165/321] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:31:06.325383 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-7cd9defe-d3bc-44f5-ac56-8ee639082c16 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:06.326863 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-7cd9defe-d3bc-44f5-ac56-8ee639082c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:06.343284 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeffc09c-b0a5-4221-b99c-097b965caa8a None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:06.400137 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 157/322] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:31:06.422270 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd9defe-d3bc-44f5-ac56-8ee639082c16 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:06.423784 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:06.444585 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeffc09c-b0a5-4221-b99c-097b965caa8a None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:06.467108 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:06.479911 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d3e5a5-29b0-4341-9cb3-c55e097754c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:06.480661 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d3e5a5-29b0-4341-9cb3-c55e097754c8 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:06.480940 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d3e5a5-29b0-4341-9cb3-c55e097754c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:06.481216 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d3e5a5-29b0-4341-9cb3-c55e097754c8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:06.483080 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d3e5a5-29b0-4341-9cb3-c55e097754c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:06.484702 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d3e5a5-29b0-4341-9cb3-c55e097754c8 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:06.486511 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 166/323] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:31:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:31:06.491562 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d3e5a5-29b0-4341-9cb3-c55e097754c8 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:06.527610 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd9defe-d3bc-44f5-ac56-8ee639082c16 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:06.568542 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaed79b-9cde-4a17-9cf2-0c9e33aa8bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:06.569349 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaed79b-9cde-4a17-9cf2-0c9e33aa8bd3 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:06.569634 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaed79b-9cde-4a17-9cf2-0c9e33aa8bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:06.569911 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaed79b-9cde-4a17-9cf2-0c9e33aa8bd3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:06.581459 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaed79b-9cde-4a17-9cf2-0c9e33aa8bd3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:31:06.589198 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-7aaed79b-9cde-4a17-9cf2-0c9e33aa8bd3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:31:06.589198 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:06.589198 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:06.589198 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:06.589198 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:06.589198 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:06.589198 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:06.589198 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:06.589198 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:06.589198 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:06.589198 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:06.589198 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:06.589198 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:31:06.589198 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:31:06.589198 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:31:06.589198 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:31:06.590062 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:31:06.590062 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:31:06.590062 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:31:06.590062 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:31:06.590062 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:06.590062 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:06.590062 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:06.590062 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:06.590062 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:06.590062 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:06.590062 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:06.590062 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:06.590062 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:06.590062 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:06.590062 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:06.590062 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:06.590861 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:06.590861 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:06.590861 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:06.590861 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:06.590861 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:31:06.590861 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:31:06.590861 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:31:06.590861 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:31:06.590861 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:31:06.590861 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:31:06.590861 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:31:06.590861 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:06.596285 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 158/324] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jun 15 14:31:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:31:06.597134 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaed79b-9cde-4a17-9cf2-0c9e33aa8bd3 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:06.601551 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d3e5a5-29b0-4341-9cb3-c55e097754c8 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:06.635894 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ebfe91-2dce-45ea-b7cc-f637b14cde40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:06.636584 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ebfe91-2dce-45ea-b7cc-f637b14cde40 None admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:06.636866 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ebfe91-2dce-45ea-b7cc-f637b14cde40 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:06.637139 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ebfe91-2dce-45ea-b7cc-f637b14cde40 None admin] PATH_INFO: `/v3/groups` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:06.637822 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ebfe91-2dce-45ea-b7cc-f637b14cde40 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:06.639777 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ebfe91-2dce-45ea-b7cc-f637b14cde40 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:06.698770 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaed79b-9cde-4a17-9cf2-0c9e33aa8bd3 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:06.700516 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:06.733999 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2821133b-209f-4d28-a55d-f1ddeb45759f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:06.734587 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2821133b-209f-4d28-a55d-f1ddeb45759f None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:06.734869 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2821133b-209f-4d28-a55d-f1ddeb45759f None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:06.735140 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2821133b-209f-4d28-a55d-f1ddeb45759f None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:06.735880 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2821133b-209f-4d28-a55d-f1ddeb45759f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:06.736219 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2821133b-209f-4d28-a55d-f1ddeb45759f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:06.744175 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 167/325] 10.222.0.42 () {60 vars in 1134 bytes} [Wed Jun 15 14:31:06 2022] POST /identity/v3/groups => generated 226 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:31:06.744373 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2821133b-209f-4d28-a55d-f1ddeb45759f None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:06.753543 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 159/326] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Jun 15 14:31:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:06.759496 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ebfe91-2dce-45ea-b7cc-f637b14cde40 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:06.770834 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:06.875162 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ebfe91-2dce-45ea-b7cc-f637b14cde40 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:06.915174 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519e23a2-673e-4f04-8090-c0aa96bdb9d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:06.916058 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e23a2-673e-4f04-8090-c0aa96bdb9d1 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:06.916394 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e23a2-673e-4f04-8090-c0aa96bdb9d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:06.916651 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e23a2-673e-4f04-8090-c0aa96bdb9d1 None admin] PATH_INFO: `/v3/users/868ddf688ff64f0f81a56f92e9c3ccf9` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:06.917866 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519e23a2-673e-4f04-8090-c0aa96bdb9d1 None admin] RBAC: Authorizing `identity:get_user(user_id=868ddf688ff64f0f81a56f92e9c3ccf9)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:06.919304 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519e23a2-673e-4f04-8090-c0aa96bdb9d1 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:06.921797 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 168/327] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jun 15 14:31:06 2022] GET /identity/v3/users/868ddf688ff64f0f81a56f92e9c3ccf9 => generated 293 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:06.927247 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2821133b-209f-4d28-a55d-f1ddeb45759f None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:07.041280 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2821133b-209f-4d28-a55d-f1ddeb45759f None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:07.078844 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d3588c-138f-4530-92f2-f49147f25a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:07.079712 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d3588c-138f-4530-92f2-f49147f25a50 None admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:07.079992 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d3588c-138f-4530-92f2-f49147f25a50 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:07.080298 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d3588c-138f-4530-92f2-f49147f25a50 None admin] PATH_INFO: `/v3/system/users/868ddf688ff64f0f81a56f92e9c3ccf9/roles/6ab04b221d1548c3b9c5941c17b9bd41` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:07.081838 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d3588c-138f-4530-92f2-f49147f25a50 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=868ddf688ff64f0f81a56f92e9c3ccf9, role_id=6ab04b221d1548c3b9c5941c17b9bd41)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:07.083242 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d3588c-138f-4530-92f2-f49147f25a50 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:07.161385 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 160/328] 10.222.0.42 () {58 vars in 1334 bytes} [Wed Jun 15 14:31:06 2022] PUT /identity/v3/system/users/868ddf688ff64f0f81a56f92e9c3ccf9/roles/6ab04b221d1548c3b9c5941c17b9bd41 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:31:07.164806 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32731c1-2e09-4f4f-bb4c-3402a1e383e2 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:07.165102 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32731c1-2e09-4f4f-bb4c-3402a1e383e2 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:07.165391 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32731c1-2e09-4f4f-bb4c-3402a1e383e2 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:07.166835 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 169/329] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:31:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:07.173910 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c431e9-026c-4c3f-ae4b-79d9e72dbcf1 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:07.174199 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c431e9-026c-4c3f-ae4b-79d9e72dbcf1 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:07.174476 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c431e9-026c-4c3f-ae4b-79d9e72dbcf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:07.192969 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-88c431e9-026c-4c3f-ae4b-79d9e72dbcf1 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:07.194326 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-88c431e9-026c-4c3f-ae4b-79d9e72dbcf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:07.222272 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:07.281769 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 161/330] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:31:07.306100 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32731c1-2e09-4f4f-bb4c-3402a1e383e2 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:07.405462 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:07.437889 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32731c1-2e09-4f4f-bb4c-3402a1e383e2 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:07.477408 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbc5f09-6104-4f6d-99a2-4537b4c4bb81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:07.478228 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc5f09-6104-4f6d-99a2-4537b4c4bb81 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:07.478496 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc5f09-6104-4f6d-99a2-4537b4c4bb81 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:07.478776 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc5f09-6104-4f6d-99a2-4537b4c4bb81 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:07.493023 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbc5f09-6104-4f6d-99a2-4537b4c4bb81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:31:07.500951 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-9bbc5f09-6104-4f6d-99a2-4537b4c4bb81 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:31:07.500951 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:07.500951 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:07.500951 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:07.500951 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:07.500951 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:07.500951 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:07.500951 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:07.500951 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:07.500951 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:07.500951 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:07.500951 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:07.500951 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:31:07.500951 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:31:07.500951 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:31:07.500951 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:31:07.501839 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:31:07.501839 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:31:07.501839 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:31:07.501839 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:31:07.501839 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:07.501839 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:07.501839 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:07.501839 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:07.501839 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:07.501839 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:07.501839 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:07.501839 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:07.501839 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:07.501839 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:07.501839 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:07.501839 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:07.502634 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:07.502634 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:07.502634 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:07.502634 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:07.502634 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:31:07.502634 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:31:07.502634 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:31:07.502634 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:31:07.502634 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:31:07.502634 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:31:07.502634 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:31:07.502634 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:07.502634 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 170/331] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jun 15 14:31:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:07.509901 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c431e9-026c-4c3f-ae4b-79d9e72dbcf1 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:07.646196 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c431e9-026c-4c3f-ae4b-79d9e72dbcf1 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:07.691354 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eebd457-6f54-4b8d-abc9-5bc046812dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:07.692029 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eebd457-6f54-4b8d-abc9-5bc046812dfe None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:07.692342 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eebd457-6f54-4b8d-abc9-5bc046812dfe None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:07.692623 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eebd457-6f54-4b8d-abc9-5bc046812dfe None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:07.693382 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eebd457-6f54-4b8d-abc9-5bc046812dfe None admin] RBAC: Adding query filter params (name=member) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:07.693693 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eebd457-6f54-4b8d-abc9-5bc046812dfe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:07.695077 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eebd457-6f54-4b8d-abc9-5bc046812dfe None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:07.707425 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 162/332] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Jun 15 14:31:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:07.712977 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbc5f09-6104-4f6d-99a2-4537b4c4bb81 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:07.866862 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbc5f09-6104-4f6d-99a2-4537b4c4bb81 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:07.911096 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea25c846-cf1e-49b8-9842-56e9be61b1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:07.912313 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea25c846-cf1e-49b8-9842-56e9be61b1ec None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:07.912788 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea25c846-cf1e-49b8-9842-56e9be61b1ec None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:07.913271 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea25c846-cf1e-49b8-9842-56e9be61b1ec None admin] PATH_INFO: `/v3/users/de0e5278ed9745c5981ffdcb67255051` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:07.915963 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea25c846-cf1e-49b8-9842-56e9be61b1ec None admin] RBAC: Authorizing `identity:get_user(user_id=de0e5278ed9745c5981ffdcb67255051)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:07.917633 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea25c846-cf1e-49b8-9842-56e9be61b1ec None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:07.920307 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 171/333] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jun 15 14:31:07 2022] GET /identity/v3/users/de0e5278ed9745c5981ffdcb67255051 => generated 293 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:07.926358 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eebd457-6f54-4b8d-abc9-5bc046812dfe None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:07.967259 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:08.072200 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eebd457-6f54-4b8d-abc9-5bc046812dfe None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:08.114441 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e373de4-ab58-4009-8091-a5928d68bb9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:08.115113 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e373de4-ab58-4009-8091-a5928d68bb9c None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:08.115396 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e373de4-ab58-4009-8091-a5928d68bb9c None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:08.115673 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e373de4-ab58-4009-8091-a5928d68bb9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:08.116543 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e373de4-ab58-4009-8091-a5928d68bb9c None admin] RBAC: Adding query filter params (role.id=9effaf496b874d01bd3f6d0555ef2d69, user.id=de0e5278ed9745c5981ffdcb67255051, scope.system=all) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:08.116858 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e373de4-ab58-4009-8091-a5928d68bb9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:08.118294 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e373de4-ab58-4009-8091-a5928d68bb9c None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:08.134614 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 163/334] 10.222.0.42 () {56 vars in 1308 bytes} [Wed Jun 15 14:31:07 2022] GET /identity/v3/role_assignments?user.id=de0e5278ed9745c5981ffdcb67255051&scope.system=all&role.id=9effaf496b874d01bd3f6d0555ef2d69 => generated 510 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:09.552888 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8753b9d1-6789-4df8-b9ba-865c68d96eac None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:09.553411 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8753b9d1-6789-4df8-b9ba-865c68d96eac None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:09.553411 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8753b9d1-6789-4df8-b9ba-865c68d96eac None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:09.554794 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 172/335] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:31:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:09.562607 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdea752-e3d2-4690-a7fa-4f65e8446110 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:09.563146 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdea752-e3d2-4690-a7fa-4f65e8446110 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:09.563146 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdea752-e3d2-4690-a7fa-4f65e8446110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:09.588479 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-1fdea752-e3d2-4690-a7fa-4f65e8446110 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:09.589697 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-1fdea752-e3d2-4690-a7fa-4f65e8446110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:09.631139 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e455ac-e954-4874-b7c7-a9c785586e44 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:09.631459 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e455ac-e954-4874-b7c7-a9c785586e44 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:09.631706 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e455ac-e954-4874-b7c7-a9c785586e44 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:09.633249 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 173/336] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:31:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:31:09.638156 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df32cca0-c751-4696-832a-ef655e4f2e7b None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:09.638847 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df32cca0-c751-4696-832a-ef655e4f2e7b None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:09.639744 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df32cca0-c751-4696-832a-ef655e4f2e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:09.655665 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:09.673730 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-df32cca0-c751-4696-832a-ef655e4f2e7b None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:09.680108 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-df32cca0-c751-4696-832a-ef655e4f2e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:09.764728 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 164/337] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:31:09.831800 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdea752-e3d2-4690-a7fa-4f65e8446110 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:09.848656 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 174/338] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:31:09.856438 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:09.887361 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df32cca0-c751-4696-832a-ef655e4f2e7b None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:10.004504 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdea752-e3d2-4690-a7fa-4f65e8446110 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:10.046880 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2a4633-47b7-4925-bb03-48d6777af89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:10.048190 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2a4633-47b7-4925-bb03-48d6777af89e None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:10.048793 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2a4633-47b7-4925-bb03-48d6777af89e None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:10.049263 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2a4633-47b7-4925-bb03-48d6777af89e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:10.064018 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2a4633-47b7-4925-bb03-48d6777af89e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:31:10.075384 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-6c2a4633-47b7-4925-bb03-48d6777af89e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:31:10.075384 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:10.075384 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:10.075384 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:10.075384 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:10.075384 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:10.075384 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:10.075384 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:10.075384 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:10.075384 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:10.075384 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:10.075384 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:10.075384 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:31:10.075384 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:31:10.075384 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:31:10.075384 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:31:10.076354 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:31:10.076354 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:31:10.076354 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:31:10.076354 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:31:10.076354 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:10.076354 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:10.076354 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:10.076354 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:10.076354 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:10.076354 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:10.076354 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:10.076354 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:10.076354 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:10.076354 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:10.076354 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:10.076354 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:10.077140 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:10.077140 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:10.077140 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:10.077140 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:10.077140 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:31:10.077140 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:31:10.077140 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:31:10.077140 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:31:10.077140 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:31:10.077140 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:31:10.077140 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:31:10.077140 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:10.084554 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 165/339] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Jun 15 14:31:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:31:10.091013 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2a4633-47b7-4925-bb03-48d6777af89e None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:10.135871 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df32cca0-c751-4696-832a-ef655e4f2e7b None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:10.178579 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:10.184450 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:10.208915 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef854697-9ad1-46a8-8ccc-85ac308c0d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:10.209927 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef854697-9ad1-46a8-8ccc-85ac308c0d51 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:10.210036 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef854697-9ad1-46a8-8ccc-85ac308c0d51 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:10.210263 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef854697-9ad1-46a8-8ccc-85ac308c0d51 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:10.232511 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2a4633-47b7-4925-bb03-48d6777af89e None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:10.232734 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef854697-9ad1-46a8-8ccc-85ac308c0d51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:31:10.242266 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-ef854697-9ad1-46a8-8ccc-85ac308c0d51 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:31:10.242266 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:10.242266 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:10.242266 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:10.242266 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:10.242266 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:10.242266 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:10.242266 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:10.242266 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:10.242266 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:10.242266 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:10.242266 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:10.242266 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:31:10.242266 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:31:10.242266 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:31:10.242266 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:31:10.243169 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:31:10.243169 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:31:10.243169 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:31:10.243169 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:31:10.243169 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:10.243169 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:10.243169 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:10.243169 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:10.243169 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:10.243169 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:10.243169 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:10.243169 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:10.243169 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:10.243169 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:10.243169 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:10.243169 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:10.243950 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:10.243950 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:10.243950 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:10.243950 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:10.243950 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:31:10.243950 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:31:10.243950 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:31:10.243950 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:31:10.243950 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:31:10.243950 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:31:10.243950 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:31:10.243950 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:10.243950 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 175/340] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jun 15 14:31:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:10.253011 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef854697-9ad1-46a8-8ccc-85ac308c0d51 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:10.289689 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5340f579-0d38-4ffe-af65-da0f5f7e0af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:10.290742 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5340f579-0d38-4ffe-af65-da0f5f7e0af2 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:10.291217 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5340f579-0d38-4ffe-af65-da0f5f7e0af2 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:10.291867 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5340f579-0d38-4ffe-af65-da0f5f7e0af2 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:10.292844 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5340f579-0d38-4ffe-af65-da0f5f7e0af2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:10.293344 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5340f579-0d38-4ffe-af65-da0f5f7e0af2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:10.295163 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5340f579-0d38-4ffe-af65-da0f5f7e0af2 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:10.304351 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 166/341] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jun 15 14:31:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:31:10.310556 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5340f579-0d38-4ffe-af65-da0f5f7e0af2 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:10.407464 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef854697-9ad1-46a8-8ccc-85ac308c0d51 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:10.456701 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5340f579-0d38-4ffe-af65-da0f5f7e0af2 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:10.468825 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa129d9c-0404-4995-ad35-3959383158b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:10.469508 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa129d9c-0404-4995-ad35-3959383158b1 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:10.469788 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa129d9c-0404-4995-ad35-3959383158b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:10.470067 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa129d9c-0404-4995-ad35-3959383158b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:10.470959 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa129d9c-0404-4995-ad35-3959383158b1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:10.471148 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa129d9c-0404-4995-ad35-3959383158b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:10.473362 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa129d9c-0404-4995-ad35-3959383158b1 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:10.481735 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 176/342] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Jun 15 14:31:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:10.487823 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa129d9c-0404-4995-ad35-3959383158b1 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:10.500115 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:10.506163 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffd9a75-e948-4c16-bf54-ed4e0dfd3e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:10.507064 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd9a75-e948-4c16-bf54-ed4e0dfd3e6d None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:10.507499 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd9a75-e948-4c16-bf54-ed4e0dfd3e6d None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:10.507927 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd9a75-e948-4c16-bf54-ed4e0dfd3e6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:10.595753 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:10.630249 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffd9a75-e948-4c16-bf54-ed4e0dfd3e6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:10.631707 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffd9a75-e948-4c16-bf54-ed4e0dfd3e6d None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:10.664327 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa129d9c-0404-4995-ad35-3959383158b1 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:10.734441 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3942e84d-8b75-480c-b2d6-6c898e5d8c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:10.735239 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3942e84d-8b75-480c-b2d6-6c898e5d8c3f None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:10.735555 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3942e84d-8b75-480c-b2d6-6c898e5d8c3f None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:10.735902 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3942e84d-8b75-480c-b2d6-6c898e5d8c3f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:10.746372 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 167/343] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jun 15 14:31:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 15 14:31:10.747783 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2e2e87-9548-4e07-87f2-b38525fb7a19 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:10.748070 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2e2e87-9548-4e07-87f2-b38525fb7a19 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:10.748370 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2e2e87-9548-4e07-87f2-b38525fb7a19 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:10.749714 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 168/344] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:31:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:10.760316 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2e2e87-9548-4e07-87f2-b38525fb7a19 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:10.797458 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:10.834619 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3942e84d-8b75-480c-b2d6-6c898e5d8c3f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:10.836285 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3942e84d-8b75-480c-b2d6-6c898e5d8c3f None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:10.891548 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2e2e87-9548-4e07-87f2-b38525fb7a19 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:10.908513 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:10.936470 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e776e5ee-9173-473c-a4e6-6eb7ae2c5d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:10.937251 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e776e5ee-9173-473c-a4e6-6eb7ae2c5d74 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:10.937528 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e776e5ee-9173-473c-a4e6-6eb7ae2c5d74 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:10.937794 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e776e5ee-9173-473c-a4e6-6eb7ae2c5d74 None admin] PATH_INFO: `/v3/projects/af60304e389b4295adfafbfc85459962` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:10.939009 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e776e5ee-9173-473c-a4e6-6eb7ae2c5d74 None admin] RBAC: Authorizing `identity:get_project(project_id=af60304e389b4295adfafbfc85459962)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:10.942175 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e776e5ee-9173-473c-a4e6-6eb7ae2c5d74 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:10.948607 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 169/345] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jun 15 14:31:10 2022] GET /identity/v3/projects/af60304e389b4295adfafbfc85459962 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:10.949911 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ea97dd-3c5c-4145-afa7-2d9ecb4e901d None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:10.950200 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ea97dd-3c5c-4145-afa7-2d9ecb4e901d None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:10.950475 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ea97dd-3c5c-4145-afa7-2d9ecb4e901d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:10.962796 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 177/346] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jun 15 14:31:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 15 14:31:10.964379 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768ef279-48d9-4c23-b01e-97a73a135cac None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:10.964667 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768ef279-48d9-4c23-b01e-97a73a135cac None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:10.964936 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768ef279-48d9-4c23-b01e-97a73a135cac None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:10.967185 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 178/347] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:31:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:10.967892 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768ef279-48d9-4c23-b01e-97a73a135cac None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:10.974301 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-04ea97dd-3c5c-4145-afa7-2d9ecb4e901d None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:10.976968 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-04ea97dd-3c5c-4145-afa7-2d9ecb4e901d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:11.066080 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 170/348] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:31:11.067559 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0786bc2e-92bb-4ae3-ab38-c93c8f9e5126 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:11.067852 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0786bc2e-92bb-4ae3-ab38-c93c8f9e5126 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:11.068431 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0786bc2e-92bb-4ae3-ab38-c93c8f9e5126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:11.094938 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-0786bc2e-92bb-4ae3-ab38-c93c8f9e5126 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:11.099242 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-0786bc2e-92bb-4ae3-ab38-c93c8f9e5126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:11.121820 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768ef279-48d9-4c23-b01e-97a73a135cac None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:11.124877 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:11.167716 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442a516a-7aae-42b4-91f3-14433b4e169a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:11.169125 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442a516a-7aae-42b4-91f3-14433b4e169a None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:11.170185 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442a516a-7aae-42b4-91f3-14433b4e169a None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:11.170668 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442a516a-7aae-42b4-91f3-14433b4e169a None admin] PATH_INFO: `/v3/groups/c1ed4738b21242e58797866b6c26ad9a` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:11.178982 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442a516a-7aae-42b4-91f3-14433b4e169a None admin] RBAC: Authorizing `identity:get_group(group_id=c1ed4738b21242e58797866b6c26ad9a)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:11.181129 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442a516a-7aae-42b4-91f3-14433b4e169a None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:11.183212 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 179/349] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Jun 15 14:31:10 2022] GET /identity/v3/groups/c1ed4738b21242e58797866b6c26ad9a => generated 226 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:11.184574 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442a516a-7aae-42b4-91f3-14433b4e169a None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:11.191964 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 171/350] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:31:11.192832 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0786bc2e-92bb-4ae3-ab38-c93c8f9e5126 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:11.224933 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:11.333870 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0786bc2e-92bb-4ae3-ab38-c93c8f9e5126 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:11.355300 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442a516a-7aae-42b4-91f3-14433b4e169a None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:11.377571 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06628d9-7e86-4f46-8d1e-dcef6eaad736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:11.378373 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06628d9-7e86-4f46-8d1e-dcef6eaad736 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:11.378875 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06628d9-7e86-4f46-8d1e-dcef6eaad736 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:11.379159 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06628d9-7e86-4f46-8d1e-dcef6eaad736 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:11.395128 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06628d9-7e86-4f46-8d1e-dcef6eaad736 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:31:11.403500 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-c06628d9-7e86-4f46-8d1e-dcef6eaad736 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:31:11.403500 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:11.403500 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:11.403500 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:11.403500 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:11.403500 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:11.403500 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:11.403500 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:11.403500 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:11.403500 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:11.403500 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:11.403500 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:11.403500 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:31:11.403500 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:31:11.403500 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:31:11.403500 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:31:11.405059 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:31:11.405059 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:31:11.405059 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:31:11.405059 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:31:11.405059 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:11.405059 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:11.405059 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:11.405059 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:11.405059 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:11.405059 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:11.405059 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:11.405059 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:11.405059 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:11.405059 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:11.405059 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:11.405059 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:11.405911 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:11.405911 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:11.405911 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:11.405911 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:11.405911 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:31:11.405911 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:31:11.405911 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:31:11.405911 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:31:11.405911 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:31:11.405911 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:31:11.405911 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:31:11.405911 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:11.405911 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 172/351] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jun 15 14:31:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:11.407260 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06628d9-7e86-4f46-8d1e-dcef6eaad736 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:11.414988 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8638b482-af5a-445d-b738-c17b6a7d6e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:11.415866 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8638b482-af5a-445d-b738-c17b6a7d6e5f None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:11.416170 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8638b482-af5a-445d-b738-c17b6a7d6e5f None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:11.416444 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8638b482-af5a-445d-b738-c17b6a7d6e5f None admin] PATH_INFO: `/v3/projects/af60304e389b4295adfafbfc85459962` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:11.417656 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8638b482-af5a-445d-b738-c17b6a7d6e5f None admin] RBAC: Authorizing `identity:get_project(project_id=af60304e389b4295adfafbfc85459962)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:11.419145 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8638b482-af5a-445d-b738-c17b6a7d6e5f None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:11.422700 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 180/352] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jun 15 14:31:11 2022] GET /identity/v3/projects/af60304e389b4295adfafbfc85459962 => generated 298 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:11.424019 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57042a9-a02e-494e-935e-63d45254f46c None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:11.424287 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57042a9-a02e-494e-935e-63d45254f46c None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:11.424567 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57042a9-a02e-494e-935e-63d45254f46c None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:11.425860 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 181/353] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:31:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:11.426600 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57042a9-a02e-494e-935e-63d45254f46c None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:11.431673 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:11.540455 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06628d9-7e86-4f46-8d1e-dcef6eaad736 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:11.549960 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:11.574617 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57042a9-a02e-494e-935e-63d45254f46c None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:11.594111 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6089c2-907c-41d8-9287-260f302a7d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:11.595047 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6089c2-907c-41d8-9287-260f302a7d1f None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:11.595484 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6089c2-907c-41d8-9287-260f302a7d1f None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:11.595934 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6089c2-907c-41d8-9287-260f302a7d1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:11.596834 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6089c2-907c-41d8-9287-260f302a7d1f None admin] RBAC: Adding query filter params (group.id=c1ed4738b21242e58797866b6c26ad9a, role.id=61e98f694ee741a9894b7e6db6dd11d3, scope.project.id=af60304e389b4295adfafbfc85459962) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:11.597150 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6089c2-907c-41d8-9287-260f302a7d1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:11.598635 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6089c2-907c-41d8-9287-260f302a7d1f None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:11.605022 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 173/354] 10.222.0.42 () {56 vars in 1376 bytes} [Wed Jun 15 14:31:11 2022] GET /identity/v3/role_assignments?group.id=c1ed4738b21242e58797866b6c26ad9a&scope.project.id=af60304e389b4295adfafbfc85459962&role.id=61e98f694ee741a9894b7e6db6dd11d3 => generated 260 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:11.606534 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6089c2-907c-41d8-9287-260f302a7d1f None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:11.628907 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47894c8-afb1-49df-91f3-159d9e12ea4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:11.629738 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47894c8-afb1-49df-91f3-159d9e12ea4b None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:11.630023 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47894c8-afb1-49df-91f3-159d9e12ea4b None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:11.630300 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47894c8-afb1-49df-91f3-159d9e12ea4b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:11.645581 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47894c8-afb1-49df-91f3-159d9e12ea4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:31:11.655863 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-c47894c8-afb1-49df-91f3-159d9e12ea4b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:31:11.655863 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:11.655863 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:11.655863 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:11.655863 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:11.655863 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:11.655863 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:11.655863 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:11.655863 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:11.655863 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:11.655863 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:11.655863 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:11.655863 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:31:11.655863 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:31:11.655863 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:31:11.655863 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:31:11.656861 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:31:11.656861 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:31:11.656861 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:31:11.656861 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:31:11.656861 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:11.656861 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:11.656861 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:11.656861 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:11.656861 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:11.656861 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:11.656861 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:11.656861 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:11.656861 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:11.656861 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:11.656861 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:11.656861 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:11.657628 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:11.657628 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:11.657628 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:11.657628 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:11.657628 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:31:11.657628 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:31:11.657628 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:31:11.657628 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:31:11.657628 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:31:11.657628 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:31:11.657628 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:31:11.657628 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:11.657628 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 182/355] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jun 15 14:31:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:11.658284 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47894c8-afb1-49df-91f3-159d9e12ea4b None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:11.714740 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:11.739041 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6089c2-907c-41d8-9287-260f302a7d1f None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:11.785487 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2d065b-1520-40d9-bd26-85a385524052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:11.786586 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2d065b-1520-40d9-bd26-85a385524052 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:11.787133 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2d065b-1520-40d9-bd26-85a385524052 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:11.787625 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2d065b-1520-40d9-bd26-85a385524052 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:11.788697 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2d065b-1520-40d9-bd26-85a385524052 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:11.789017 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2d065b-1520-40d9-bd26-85a385524052 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:11.790437 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2d065b-1520-40d9-bd26-85a385524052 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:11.803896 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 174/356] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Jun 15 14:31:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:11.805538 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e837b7df-cbc4-4230-8a2f-438c30513164 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:11.806067 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e837b7df-cbc4-4230-8a2f-438c30513164 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:11.806560 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e837b7df-cbc4-4230-8a2f-438c30513164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:11.812106 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47894c8-afb1-49df-91f3-159d9e12ea4b None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:11.829915 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-e837b7df-cbc4-4230-8a2f-438c30513164 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:11.831486 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e837b7df-cbc4-4230-8a2f-438c30513164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:11.863199 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2533e6d2-049c-4849-9d12-9cafbc2429c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:11.864137 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533e6d2-049c-4849-9d12-9cafbc2429c2 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:11.864427 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533e6d2-049c-4849-9d12-9cafbc2429c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:11.864703 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533e6d2-049c-4849-9d12-9cafbc2429c2 None admin] PATH_INFO: `/v3/groups/c1ed4738b21242e58797866b6c26ad9a` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:11.867128 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2533e6d2-049c-4849-9d12-9cafbc2429c2 None admin] RBAC: Authorizing `identity:get_group(group_id=c1ed4738b21242e58797866b6c26ad9a)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:11.868698 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2533e6d2-049c-4849-9d12-9cafbc2429c2 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:11.870719 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 183/357] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Jun 15 14:31:11 2022] GET /identity/v3/groups/c1ed4738b21242e58797866b6c26ad9a => generated 226 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:11.871558 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2533e6d2-049c-4849-9d12-9cafbc2429c2 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:11.893404 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:11.933951 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 175/358] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:31:11.935704 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e837b7df-cbc4-4230-8a2f-438c30513164 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:12.015297 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2533e6d2-049c-4849-9d12-9cafbc2429c2 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:12.032605 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:12.062320 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79586727-95d0-47fa-b15d-b9e4bf939b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:12.062930 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79586727-95d0-47fa-b15d-b9e4bf939b39 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:12.063216 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79586727-95d0-47fa-b15d-b9e4bf939b39 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:12.063498 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79586727-95d0-47fa-b15d-b9e4bf939b39 None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:12.063591 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e837b7df-cbc4-4230-8a2f-438c30513164 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:12.064348 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79586727-95d0-47fa-b15d-b9e4bf939b39 None admin] RBAC: Adding query filter params (name=member) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:12.064665 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79586727-95d0-47fa-b15d-b9e4bf939b39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:12.066237 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79586727-95d0-47fa-b15d-b9e4bf939b39 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:12.079743 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 184/359] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Jun 15 14:31:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:12.080688 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79586727-95d0-47fa-b15d-b9e4bf939b39 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:12.112293 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7298adac-20df-44c1-8bca-b5c5eb441a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:12.113210 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7298adac-20df-44c1-8bca-b5c5eb441a52 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:12.113497 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7298adac-20df-44c1-8bca-b5c5eb441a52 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:12.114054 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7298adac-20df-44c1-8bca-b5c5eb441a52 None admin] PATH_INFO: `/v3/users/868ddf688ff64f0f81a56f92e9c3ccf9` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:12.115303 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7298adac-20df-44c1-8bca-b5c5eb441a52 None admin] RBAC: Authorizing `identity:get_user(user_id=868ddf688ff64f0f81a56f92e9c3ccf9)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:12.116980 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7298adac-20df-44c1-8bca-b5c5eb441a52 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:12.119528 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 176/360] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jun 15 14:31:11 2022] GET /identity/v3/users/868ddf688ff64f0f81a56f92e9c3ccf9 => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:12.120947 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4acfd1c-0ee7-4433-a219-57fb25a3191a None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:12.121388 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4acfd1c-0ee7-4433-a219-57fb25a3191a None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:12.121818 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4acfd1c-0ee7-4433-a219-57fb25a3191a None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:12.123924 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 177/361] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:31:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:12.124876 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4acfd1c-0ee7-4433-a219-57fb25a3191a None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:12.196351 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:12.235902 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79586727-95d0-47fa-b15d-b9e4bf939b39 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:12.258648 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4acfd1c-0ee7-4433-a219-57fb25a3191a None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:12.286716 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164c5db2-e23e-42b9-96d0-1d5c343f5554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:12.286716 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164c5db2-e23e-42b9-96d0-1d5c343f5554 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:12.286716 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164c5db2-e23e-42b9-96d0-1d5c343f5554 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:12.286716 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164c5db2-e23e-42b9-96d0-1d5c343f5554 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:12.287132 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164c5db2-e23e-42b9-96d0-1d5c343f5554 None admin] RBAC: Adding query filter params (group.id=c1ed4738b21242e58797866b6c26ad9a, role.id=9effaf496b874d01bd3f6d0555ef2d69, scope.project.id=af60304e389b4295adfafbfc85459962) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:12.287132 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164c5db2-e23e-42b9-96d0-1d5c343f5554 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:12.288047 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164c5db2-e23e-42b9-96d0-1d5c343f5554 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:12.297543 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 185/362] 10.222.0.42 () {56 vars in 1376 bytes} [Wed Jun 15 14:31:12 2022] GET /identity/v3/role_assignments?group.id=c1ed4738b21242e58797866b6c26ad9a&scope.project.id=af60304e389b4295adfafbfc85459962&role.id=9effaf496b874d01bd3f6d0555ef2d69 => generated 260 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:12.298639 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164c5db2-e23e-42b9-96d0-1d5c343f5554 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:12.318727 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da39b720-1481-45c8-ab8a-ef3eeaa8fbbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:12.319476 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da39b720-1481-45c8-ab8a-ef3eeaa8fbbe None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:12.319755 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da39b720-1481-45c8-ab8a-ef3eeaa8fbbe None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:12.320031 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da39b720-1481-45c8-ab8a-ef3eeaa8fbbe None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:12.336647 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da39b720-1481-45c8-ab8a-ef3eeaa8fbbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:31:12.349092 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-da39b720-1481-45c8-ab8a-ef3eeaa8fbbe None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:31:12.349092 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:12.349092 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:12.349092 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:12.349092 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:12.349092 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:12.349092 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:12.349092 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:12.349092 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:12.349092 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:12.349092 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:12.349092 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:12.349092 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:31:12.349092 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:31:12.349092 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:31:12.349092 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:31:12.350091 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:31:12.350091 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:31:12.350091 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:31:12.350091 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:31:12.350091 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:12.350091 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:12.350091 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:12.350091 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:12.350091 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:12.350091 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:12.350091 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:12.350091 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:12.350091 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:12.350091 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:12.350091 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:12.350091 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:12.350903 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:12.350903 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:12.350903 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:12.350903 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:12.350903 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:31:12.350903 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:31:12.350903 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:31:12.350903 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:31:12.350903 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:31:12.350903 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:31:12.350903 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:31:12.350903 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:12.359265 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 178/363] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Jun 15 14:31:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:12.360541 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da39b720-1481-45c8-ab8a-ef3eeaa8fbbe None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:12.366162 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:12.446343 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164c5db2-e23e-42b9-96d0-1d5c343f5554 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:12.485759 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da39b720-1481-45c8-ab8a-ef3eeaa8fbbe None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:12.493328 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9531e10-911e-4949-bb48-643802dd8adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:12.494833 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9531e10-911e-4949-bb48-643802dd8adc None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:12.495326 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9531e10-911e-4949-bb48-643802dd8adc None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:12.495823 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9531e10-911e-4949-bb48-643802dd8adc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:12.513231 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:12.530953 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d85aea2-3a74-46ef-901f-349e300ea50a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:12.531617 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d85aea2-3a74-46ef-901f-349e300ea50a None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:12.531898 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d85aea2-3a74-46ef-901f-349e300ea50a None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:12.533071 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d85aea2-3a74-46ef-901f-349e300ea50a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:12.533609 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d85aea2-3a74-46ef-901f-349e300ea50a None admin] RBAC: Adding query filter params (role.id=6ab04b221d1548c3b9c5941c17b9bd41, user.id=868ddf688ff64f0f81a56f92e9c3ccf9, scope.system=all) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:12.533823 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d85aea2-3a74-46ef-901f-349e300ea50a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:12.535395 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d85aea2-3a74-46ef-901f-349e300ea50a None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:12.554464 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 179/364] 10.222.0.42 () {56 vars in 1308 bytes} [Wed Jun 15 14:31:12 2022] GET /identity/v3/role_assignments?user.id=868ddf688ff64f0f81a56f92e9c3ccf9&scope.system=all&role.id=6ab04b221d1548c3b9c5941c17b9bd41 => generated 510 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:12.555802 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7673e646-5730-4c94-a163-6d2adacb9a4a None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:12.556111 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7673e646-5730-4c94-a163-6d2adacb9a4a None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:12.556417 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7673e646-5730-4c94-a163-6d2adacb9a4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:12.576649 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-7673e646-5730-4c94-a163-6d2adacb9a4a None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:12.578149 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-7673e646-5730-4c94-a163-6d2adacb9a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:12.600663 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9531e10-911e-4949-bb48-643802dd8adc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:12.602179 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9531e10-911e-4949-bb48-643802dd8adc None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:12.662125 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:12.686933 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 180/365] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:31:12.687806 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7673e646-5730-4c94-a163-6d2adacb9a4a None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:12.728595 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 186/366] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jun 15 14:31:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:31:12.734946 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9531e10-911e-4949-bb48-643802dd8adc None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:12.785638 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:12.818272 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7673e646-5730-4c94-a163-6d2adacb9a4a None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:12.854606 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9531e10-911e-4949-bb48-643802dd8adc None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:12.859208 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9e4902-912f-4dfa-9785-93d402161bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:12.859900 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9e4902-912f-4dfa-9785-93d402161bb8 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:12.860213 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9e4902-912f-4dfa-9785-93d402161bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:12.860493 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9e4902-912f-4dfa-9785-93d402161bb8 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:12.861253 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9e4902-912f-4dfa-9785-93d402161bb8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:12.861560 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9e4902-912f-4dfa-9785-93d402161bb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:12.862955 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9e4902-912f-4dfa-9785-93d402161bb8 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:12.873091 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 181/367] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jun 15 14:31:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:12.874037 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9e4902-912f-4dfa-9785-93d402161bb8 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:12.900061 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5acaa5-0338-4534-a9e6-d269b30a4358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:12.901313 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5acaa5-0338-4534-a9e6-d269b30a4358 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:12.901796 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5acaa5-0338-4534-a9e6-d269b30a4358 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:12.902412 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5acaa5-0338-4534-a9e6-d269b30a4358 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:12.915217 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5acaa5-0338-4534-a9e6-d269b30a4358 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:31:12.926082 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-2f5acaa5-0338-4534-a9e6-d269b30a4358 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:31:12.926082 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:12.926082 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:12.926082 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:12.926082 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:12.926082 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:12.926082 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:12.926082 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:12.926082 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:12.926082 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:12.926082 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:12.926082 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:12.926082 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:31:12.926082 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:31:12.926082 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:31:12.926082 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:31:12.927006 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:31:12.927006 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:31:12.927006 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:31:12.927006 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:31:12.927006 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:12.927006 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:12.927006 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:12.927006 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:12.927006 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:12.927006 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:12.927006 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:12.927006 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:12.927006 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:12.927006 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:12.927006 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:12.927006 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:12.927796 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:12.927796 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:12.927796 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:12.927796 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:12.927796 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:31:12.927796 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:31:12.927796 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:31:12.927796 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:31:12.927796 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:31:12.927796 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:31:12.927796 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:31:12.927796 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:12.930763 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 187/368] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jun 15 14:31:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:12.931738 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5acaa5-0338-4534-a9e6-d269b30a4358 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:12.944916 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:13.016893 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9e4902-912f-4dfa-9785-93d402161bb8 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:13.067015 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7820b71-116a-483b-b838-0c534951ec37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:13.068784 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7820b71-116a-483b-b838-0c534951ec37 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:13.069277 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7820b71-116a-483b-b838-0c534951ec37 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:13.070052 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7820b71-116a-483b-b838-0c534951ec37 None admin] PATH_INFO: `/v3/projects/e58ea743785943a9936197cc15d05a80` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:13.071701 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7820b71-116a-483b-b838-0c534951ec37 None admin] RBAC: Authorizing `identity:get_project(project_id=e58ea743785943a9936197cc15d05a80)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:13.075006 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7820b71-116a-483b-b838-0c534951ec37 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:13.077867 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 182/369] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jun 15 14:31:12 2022] GET /identity/v3/projects/e58ea743785943a9936197cc15d05a80 => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:13.077867 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7820b71-116a-483b-b838-0c534951ec37 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:13.087510 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:13.094399 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5acaa5-0338-4534-a9e6-d269b30a4358 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:13.132390 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296a2885-34ee-4f2a-afb1-8ccc98a8bdee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:13.133381 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296a2885-34ee-4f2a-afb1-8ccc98a8bdee None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:13.133882 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296a2885-34ee-4f2a-afb1-8ccc98a8bdee None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:13.134390 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296a2885-34ee-4f2a-afb1-8ccc98a8bdee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:13.197217 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7820b71-116a-483b-b838-0c534951ec37 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:13.208852 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296a2885-34ee-4f2a-afb1-8ccc98a8bdee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:13.210697 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296a2885-34ee-4f2a-afb1-8ccc98a8bdee None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:13.219135 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:13.235354 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c03e6a-b934-405d-b43c-44801bfc76fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:13.236366 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c03e6a-b934-405d-b43c-44801bfc76fc None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:13.236610 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c03e6a-b934-405d-b43c-44801bfc76fc None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:13.236886 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c03e6a-b934-405d-b43c-44801bfc76fc None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:13.237743 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c03e6a-b934-405d-b43c-44801bfc76fc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:13.238052 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c03e6a-b934-405d-b43c-44801bfc76fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:13.239431 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c03e6a-b934-405d-b43c-44801bfc76fc None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:13.251700 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 183/370] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jun 15 14:31:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:13.252425 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c03e6a-b934-405d-b43c-44801bfc76fc None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:13.320579 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:13.324573 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 188/371] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jun 15 14:31:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:31:13.325661 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296a2885-34ee-4f2a-afb1-8ccc98a8bdee None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:13.370489 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c03e6a-b934-405d-b43c-44801bfc76fc None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:13.409319 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f705e51-1e30-4483-907e-bbbdfc4d5e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:13.410246 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f705e51-1e30-4483-907e-bbbdfc4d5e76 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:13.410473 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f705e51-1e30-4483-907e-bbbdfc4d5e76 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:13.410749 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f705e51-1e30-4483-907e-bbbdfc4d5e76 None admin] PATH_INFO: `/v3/groups/c1ed4738b21242e58797866b6c26ad9a` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:13.411912 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f705e51-1e30-4483-907e-bbbdfc4d5e76 None admin] RBAC: Authorizing `identity:get_group(group_id=c1ed4738b21242e58797866b6c26ad9a)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:13.413357 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f705e51-1e30-4483-907e-bbbdfc4d5e76 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:13.415192 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 184/372] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Jun 15 14:31:13 2022] GET /identity/v3/groups/c1ed4738b21242e58797866b6c26ad9a => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:13.416005 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f705e51-1e30-4483-907e-bbbdfc4d5e76 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:13.435484 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296a2885-34ee-4f2a-afb1-8ccc98a8bdee None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:13.471819 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4b5bf3-5323-4f50-96e8-63b443cdc237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:13.472843 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4b5bf3-5323-4f50-96e8-63b443cdc237 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:13.473045 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:13.473542 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4b5bf3-5323-4f50-96e8-63b443cdc237 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:13.474039 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4b5bf3-5323-4f50-96e8-63b443cdc237 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:13.529944 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f705e51-1e30-4483-907e-bbbdfc4d5e76 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:13.546497 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:13.549066 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4b5bf3-5323-4f50-96e8-63b443cdc237 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:13.550850 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4b5bf3-5323-4f50-96e8-63b443cdc237 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:13.571265 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2e8b18-6d25-4ab2-be0e-2ef49bc610bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:13.572136 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e8b18-6d25-4ab2-be0e-2ef49bc610bf None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:13.572631 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e8b18-6d25-4ab2-be0e-2ef49bc610bf None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:13.572933 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e8b18-6d25-4ab2-be0e-2ef49bc610bf None admin] PATH_INFO: `/v3/projects/e58ea743785943a9936197cc15d05a80` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:13.574181 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2e8b18-6d25-4ab2-be0e-2ef49bc610bf None admin] RBAC: Authorizing `identity:get_project(project_id=e58ea743785943a9936197cc15d05a80)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:13.576420 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2e8b18-6d25-4ab2-be0e-2ef49bc610bf None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:13.578721 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 185/373] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jun 15 14:31:13 2022] GET /identity/v3/projects/e58ea743785943a9936197cc15d05a80 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:13.583598 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2e8b18-6d25-4ab2-be0e-2ef49bc610bf None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:13.705477 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 189/374] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jun 15 14:31:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:31:13.705477 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4b5bf3-5323-4f50-96e8-63b443cdc237 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:13.732927 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2e8b18-6d25-4ab2-be0e-2ef49bc610bf None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:13.756632 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:13.777334 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ea8c19-d3e6-4ec8-8fb8-98d9f9f2af47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:13.778000 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea8c19-d3e6-4ec8-8fb8-98d9f9f2af47 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:13.778280 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea8c19-d3e6-4ec8-8fb8-98d9f9f2af47 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:13.779074 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea8c19-d3e6-4ec8-8fb8-98d9f9f2af47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:13.780374 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ea8c19-d3e6-4ec8-8fb8-98d9f9f2af47 None admin] RBAC: Adding query filter params (group.id=c1ed4738b21242e58797866b6c26ad9a, role.id=9effaf496b874d01bd3f6d0555ef2d69, scope.project.id=e58ea743785943a9936197cc15d05a80) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:13.780727 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ea8c19-d3e6-4ec8-8fb8-98d9f9f2af47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:13.782146 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ea8c19-d3e6-4ec8-8fb8-98d9f9f2af47 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:13.788568 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 186/375] 10.222.0.42 () {56 vars in 1376 bytes} [Wed Jun 15 14:31:13 2022] GET /identity/v3/role_assignments?group.id=c1ed4738b21242e58797866b6c26ad9a&scope.project.id=e58ea743785943a9936197cc15d05a80&role.id=9effaf496b874d01bd3f6d0555ef2d69 => generated 260 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:13.789397 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ea8c19-d3e6-4ec8-8fb8-98d9f9f2af47 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:13.889876 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:13.895417 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4b5bf3-5323-4f50-96e8-63b443cdc237 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:13.943529 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ea8c19-d3e6-4ec8-8fb8-98d9f9f2af47 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:13.993653 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff61c09-36f2-4011-8ee0-32b6c7134a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:13.994460 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff61c09-36f2-4011-8ee0-32b6c7134a62 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:13.994942 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff61c09-36f2-4011-8ee0-32b6c7134a62 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:13.995219 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff61c09-36f2-4011-8ee0-32b6c7134a62 None admin] PATH_INFO: `/v3/projects/688b80ae31764832aed9d6e0f3d3660f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:13.996792 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827bd87f-32c4-4a27-9889-1416702537dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:13.997014 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff61c09-36f2-4011-8ee0-32b6c7134a62 None admin] RBAC: Authorizing `identity:get_project(project_id=688b80ae31764832aed9d6e0f3d3660f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:13.998220 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827bd87f-32c4-4a27-9889-1416702537dd None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:13.998500 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827bd87f-32c4-4a27-9889-1416702537dd None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:13.998778 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827bd87f-32c4-4a27-9889-1416702537dd None admin] PATH_INFO: `/v3/groups/c1ed4738b21242e58797866b6c26ad9a` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:13.999440 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff61c09-36f2-4011-8ee0-32b6c7134a62 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:14.000011 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827bd87f-32c4-4a27-9889-1416702537dd None admin] RBAC: Authorizing `identity:get_group(group_id=c1ed4738b21242e58797866b6c26ad9a)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:14.001470 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827bd87f-32c4-4a27-9889-1416702537dd None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:14.001585 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 187/376] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jun 15 14:31:13 2022] GET /identity/v3/projects/688b80ae31764832aed9d6e0f3d3660f => generated 344 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:14.003165 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 190/377] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Jun 15 14:31:13 2022] GET /identity/v3/groups/c1ed4738b21242e58797866b6c26ad9a => generated 226 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:14.008789 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff61c09-36f2-4011-8ee0-32b6c7134a62 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:14.008917 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827bd87f-32c4-4a27-9889-1416702537dd None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:14.071391 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:14.137220 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff61c09-36f2-4011-8ee0-32b6c7134a62 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:14.149265 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827bd87f-32c4-4a27-9889-1416702537dd None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:14.179590 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898bd01f-5019-4fd3-aeaa-57a235a92808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:14.180527 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898bd01f-5019-4fd3-aeaa-57a235a92808 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:14.180808 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898bd01f-5019-4fd3-aeaa-57a235a92808 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:14.181082 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898bd01f-5019-4fd3-aeaa-57a235a92808 None admin] PATH_INFO: `/v3/groups/ac0958f6024f49e2ab5ba11855c918bf` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:14.189001 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898bd01f-5019-4fd3-aeaa-57a235a92808 None admin] RBAC: Authorizing `identity:get_group(group_id=ac0958f6024f49e2ab5ba11855c918bf)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:14.190662 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898bd01f-5019-4fd3-aeaa-57a235a92808 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:14.192745 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 188/378] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Jun 15 14:31:14 2022] GET /identity/v3/groups/ac0958f6024f49e2ab5ba11855c918bf => generated 229 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:14.201116 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898bd01f-5019-4fd3-aeaa-57a235a92808 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:14.201366 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b278d2ee-7f35-4404-8670-5f07de437b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:14.202184 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b278d2ee-7f35-4404-8670-5f07de437b7f None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:14.202471 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b278d2ee-7f35-4404-8670-5f07de437b7f None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:14.202749 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b278d2ee-7f35-4404-8670-5f07de437b7f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:14.203590 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b278d2ee-7f35-4404-8670-5f07de437b7f None admin] RBAC: Adding query filter params (group.id=c1ed4738b21242e58797866b6c26ad9a, role.id=61e98f694ee741a9894b7e6db6dd11d3, scope.project.id=e58ea743785943a9936197cc15d05a80) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:14.203903 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b278d2ee-7f35-4404-8670-5f07de437b7f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:14.205375 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b278d2ee-7f35-4404-8670-5f07de437b7f None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:14.213991 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 191/379] 10.222.0.42 () {56 vars in 1376 bytes} [Wed Jun 15 14:31:14 2022] GET /identity/v3/role_assignments?group.id=c1ed4738b21242e58797866b6c26ad9a&scope.project.id=e58ea743785943a9936197cc15d05a80&role.id=61e98f694ee741a9894b7e6db6dd11d3 => generated 260 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:14.366815 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf65918-dfa0-444e-b8c2-8b4a2e938622 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:14.367305 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf65918-dfa0-444e-b8c2-8b4a2e938622 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:14.367694 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf65918-dfa0-444e-b8c2-8b4a2e938622 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:14.370444 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 192/380] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:31:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:31:14.374704 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898bd01f-5019-4fd3-aeaa-57a235a92808 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:14.376041 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872fa7ae-4842-40fa-b86d-ffd7e3c3ffae None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:14.376362 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872fa7ae-4842-40fa-b86d-ffd7e3c3ffae None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:14.376648 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872fa7ae-4842-40fa-b86d-ffd7e3c3ffae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:14.396166 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:14.408168 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-872fa7ae-4842-40fa-b86d-ffd7e3c3ffae None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:14.409388 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-872fa7ae-4842-40fa-b86d-ffd7e3c3ffae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:14.431387 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab9f700-0a73-4d0e-ba55-054d90ccff94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:14.432078 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab9f700-0a73-4d0e-ba55-054d90ccff94 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:14.432787 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab9f700-0a73-4d0e-ba55-054d90ccff94 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:14.433065 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab9f700-0a73-4d0e-ba55-054d90ccff94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:14.434038 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab9f700-0a73-4d0e-ba55-054d90ccff94 None admin] RBAC: Adding query filter params (group.id=ac0958f6024f49e2ab5ba11855c918bf, role.id=2bd1979f53bb4853a50dd5bee6aa0782, scope.project.id=688b80ae31764832aed9d6e0f3d3660f) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:14.434353 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab9f700-0a73-4d0e-ba55-054d90ccff94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:14.435876 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab9f700-0a73-4d0e-ba55-054d90ccff94 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:14.448824 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 189/381] 10.222.0.42 () {56 vars in 1376 bytes} [Wed Jun 15 14:31:14 2022] GET /identity/v3/role_assignments?group.id=ac0958f6024f49e2ab5ba11855c918bf&scope.project.id=688b80ae31764832aed9d6e0f3d3660f&role.id=2bd1979f53bb4853a50dd5bee6aa0782 => generated 260 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:14.453124 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:14.516017 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 193/382] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:31:14.562374 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab9f700-0a73-4d0e-ba55-054d90ccff94 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:14.700556 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab9f700-0a73-4d0e-ba55-054d90ccff94 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:14.739708 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddb8fb3-6a41-4910-95d4-fa9014816b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:14.740914 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb8fb3-6a41-4910-95d4-fa9014816b4c None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:14.741417 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb8fb3-6a41-4910-95d4-fa9014816b4c None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:14.741925 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb8fb3-6a41-4910-95d4-fa9014816b4c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:14.755461 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddb8fb3-6a41-4910-95d4-fa9014816b4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:31:14.764676 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-6ddb8fb3-6a41-4910-95d4-fa9014816b4c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:31:14.764676 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:14.764676 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:14.764676 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:14.764676 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:14.764676 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:14.764676 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:14.764676 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:14.764676 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:14.764676 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:14.764676 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:14.764676 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:14.764676 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:31:14.764676 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:31:14.764676 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:31:14.764676 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:31:14.765567 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:31:14.765567 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:31:14.765567 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:31:14.765567 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:31:14.765567 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:14.765567 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:14.765567 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:14.765567 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:14.765567 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:14.765567 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:14.765567 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:14.765567 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:14.765567 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:14.765567 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:14.765567 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:14.765567 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:14.766367 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:14.766367 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:14.766367 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:14.766367 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:14.766367 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:31:14.766367 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:31:14.766367 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:31:14.766367 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:31:14.766367 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:31:14.766367 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:31:14.766367 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:31:14.766367 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:14.770254 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 190/383] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Jun 15 14:31:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:14.776262 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872fa7ae-4842-40fa-b86d-ffd7e3c3ffae None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:14.928069 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872fa7ae-4842-40fa-b86d-ffd7e3c3ffae None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:14.981057 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1544ecf1-1f32-49fd-9c21-c8b3cb392e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:14.982211 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1544ecf1-1f32-49fd-9c21-c8b3cb392e42 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:14.982687 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1544ecf1-1f32-49fd-9c21-c8b3cb392e42 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:14.983003 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1544ecf1-1f32-49fd-9c21-c8b3cb392e42 None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:14.983687 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7f37ff-2d3b-432a-9d59-1dbd7eff831a None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:14.984154 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1544ecf1-1f32-49fd-9c21-c8b3cb392e42 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:14.984241 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7f37ff-2d3b-432a-9d59-1dbd7eff831a None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:14.984318 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1544ecf1-1f32-49fd-9c21-c8b3cb392e42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:14.984393 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7f37ff-2d3b-432a-9d59-1dbd7eff831a None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:14.985437 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1544ecf1-1f32-49fd-9c21-c8b3cb392e42 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:14.989760 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 191/384] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:31:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:14.992377 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99c65bb-9c9f-432c-bea3-d90dae82474f None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:14.992667 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99c65bb-9c9f-432c-bea3-d90dae82474f None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:14.992948 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99c65bb-9c9f-432c-bea3-d90dae82474f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:15.001893 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 194/385] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jun 15 14:31:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:15.004574 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1544ecf1-1f32-49fd-9c21-c8b3cb392e42 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:15.024752 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-d99c65bb-9c9f-432c-bea3-d90dae82474f None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:15.025971 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-d99c65bb-9c9f-432c-bea3-d90dae82474f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:15.053645 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:15.090548 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:15.140916 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 192/386] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:31:15.182315 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1544ecf1-1f32-49fd-9c21-c8b3cb392e42 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:15.188404 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99c65bb-9c9f-432c-bea3-d90dae82474f None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:15.248067 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ee80f7-e134-48d2-9695-fec5d66382af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:15.249002 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee80f7-e134-48d2-9695-fec5d66382af None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:15.249281 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee80f7-e134-48d2-9695-fec5d66382af None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:15.249555 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee80f7-e134-48d2-9695-fec5d66382af None admin] PATH_INFO: `/v3/groups/c1ed4738b21242e58797866b6c26ad9a` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:15.252560 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ee80f7-e134-48d2-9695-fec5d66382af None admin] RBAC: Authorizing `identity:get_group(group_id=c1ed4738b21242e58797866b6c26ad9a)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:15.254064 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ee80f7-e134-48d2-9695-fec5d66382af None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:15.255877 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 195/387] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Jun 15 14:31:15 2022] GET /identity/v3/groups/c1ed4738b21242e58797866b6c26ad9a => generated 226 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:31:15.262672 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ee80f7-e134-48d2-9695-fec5d66382af None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:15.365209 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99c65bb-9c9f-432c-bea3-d90dae82474f None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:15.422788 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e21589e-17b2-4412-b723-cc887176cfb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:15.423668 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e21589e-17b2-4412-b723-cc887176cfb1 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:15.423952 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e21589e-17b2-4412-b723-cc887176cfb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:15.424250 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e21589e-17b2-4412-b723-cc887176cfb1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:15.447978 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e21589e-17b2-4412-b723-cc887176cfb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:31:15.457079 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:15.467839 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-2e21589e-17b2-4412-b723-cc887176cfb1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:31:15.467839 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:15.467839 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:15.467839 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:15.467839 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:15.467839 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:15.467839 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:15.467839 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:15.467839 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:15.467839 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:15.467839 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:15.467839 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:15.467839 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:31:15.467839 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:31:15.467839 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:31:15.467839 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:31:15.472674 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:31:15.472674 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:31:15.472674 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:31:15.472674 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:31:15.472674 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:15.472674 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:15.472674 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:15.472674 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:15.472674 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:15.472674 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:15.472674 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:15.472674 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:15.472674 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:15.472674 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:15.472674 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:15.472674 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:15.473469 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:15.473469 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:15.473469 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:15.473469 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:15.473469 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:31:15.473469 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:31:15.473469 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:31:15.473469 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:31:15.473469 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:31:15.473469 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:31:15.473469 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:31:15.473469 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:15.473469 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 193/388] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jun 15 14:31:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:15.474180 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ee80f7-e134-48d2-9695-fec5d66382af None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:15.475425 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e21589e-17b2-4412-b723-cc887176cfb1 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:15.526771 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:15.556552 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf38d6a-45c9-4de0-aaf9-f65adb61502f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:15.557665 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf38d6a-45c9-4de0-aaf9-f65adb61502f None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:15.558304 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf38d6a-45c9-4de0-aaf9-f65adb61502f None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:15.558304 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf38d6a-45c9-4de0-aaf9-f65adb61502f None admin] PATH_INFO: `/v3/projects/af60304e389b4295adfafbfc85459962` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:15.561070 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf38d6a-45c9-4de0-aaf9-f65adb61502f None admin] RBAC: Authorizing `identity:get_project(project_id=af60304e389b4295adfafbfc85459962)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:15.562509 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf38d6a-45c9-4de0-aaf9-f65adb61502f None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:15.564307 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 196/389] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jun 15 14:31:15 2022] GET /identity/v3/projects/af60304e389b4295adfafbfc85459962 => generated 298 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:31:15.575920 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf38d6a-45c9-4de0-aaf9-f65adb61502f None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:15.660448 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e21589e-17b2-4412-b723-cc887176cfb1 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:15.730249 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d188c3be-7487-4d88-a219-177476f19710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:15.730925 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d188c3be-7487-4d88-a219-177476f19710 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:15.732444 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d188c3be-7487-4d88-a219-177476f19710 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:15.732727 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d188c3be-7487-4d88-a219-177476f19710 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:15.734856 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d188c3be-7487-4d88-a219-177476f19710 None admin] RBAC: Adding query filter params (name=member) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:15.735169 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d188c3be-7487-4d88-a219-177476f19710 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:15.736573 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d188c3be-7487-4d88-a219-177476f19710 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:15.756794 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 194/390] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Jun 15 14:31:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:15.759655 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d188c3be-7487-4d88-a219-177476f19710 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:15.768400 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf38d6a-45c9-4de0-aaf9-f65adb61502f None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:15.820872 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a270e-3d7a-4eab-b9c8-a0a02c2d5e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:15.821682 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a270e-3d7a-4eab-b9c8-a0a02c2d5e68 None admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:15.821966 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a270e-3d7a-4eab-b9c8-a0a02c2d5e68 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:15.822243 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a270e-3d7a-4eab-b9c8-a0a02c2d5e68 None admin] PATH_INFO: `/v3/projects/af60304e389b4295adfafbfc85459962/groups/c1ed4738b21242e58797866b6c26ad9a/roles/61e98f694ee741a9894b7e6db6dd11d3` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:15.828880 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825a270e-3d7a-4eab-b9c8-a0a02c2d5e68 None admin] RBAC: Authorizing `identity:create_grant(project_id=af60304e389b4295adfafbfc85459962, group_id=c1ed4738b21242e58797866b6c26ad9a, role_id=61e98f694ee741a9894b7e6db6dd11d3)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:15.830380 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825a270e-3d7a-4eab-b9c8-a0a02c2d5e68 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:15.871633 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:15.882970 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d188c3be-7487-4d88-a219-177476f19710 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:15.915217 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 197/391] 10.222.0.42 () {58 vars in 1442 bytes} [Wed Jun 15 14:31:15 2022] PUT /identity/v3/projects/af60304e389b4295adfafbfc85459962/groups/c1ed4738b21242e58797866b6c26ad9a/roles/61e98f694ee741a9894b7e6db6dd11d3 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:31:15.930264 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2742c8aa-00cb-43ef-ad9d-8d0b53d8b925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:15.931336 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2742c8aa-00cb-43ef-ad9d-8d0b53d8b925 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:15.931621 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2742c8aa-00cb-43ef-ad9d-8d0b53d8b925 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:15.931953 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2742c8aa-00cb-43ef-ad9d-8d0b53d8b925 None admin] PATH_INFO: `/v3/groups/c1ed4738b21242e58797866b6c26ad9a` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:15.933318 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2742c8aa-00cb-43ef-ad9d-8d0b53d8b925 None admin] RBAC: Authorizing `identity:get_group(group_id=c1ed4738b21242e58797866b6c26ad9a)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:15.934795 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2742c8aa-00cb-43ef-ad9d-8d0b53d8b925 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:15.940027 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 195/392] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Jun 15 14:31:15 2022] GET /identity/v3/groups/c1ed4738b21242e58797866b6c26ad9a => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:15.943776 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a270e-3d7a-4eab-b9c8-a0a02c2d5e68 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:15.971460 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:16.058052 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a270e-3d7a-4eab-b9c8-a0a02c2d5e68 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:16.094965 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd5fffb-c850-46ef-a921-76ee2322c3e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:16.095757 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd5fffb-c850-46ef-a921-76ee2322c3e6 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:16.096039 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd5fffb-c850-46ef-a921-76ee2322c3e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:16.096352 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd5fffb-c850-46ef-a921-76ee2322c3e6 None admin] PATH_INFO: `/v3/projects/af60304e389b4295adfafbfc85459962` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:16.097658 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd5fffb-c850-46ef-a921-76ee2322c3e6 None admin] RBAC: Authorizing `identity:get_project(project_id=af60304e389b4295adfafbfc85459962)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:16.099177 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd5fffb-c850-46ef-a921-76ee2322c3e6 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:16.105834 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 198/393] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jun 15 14:31:15 2022] GET /identity/v3/projects/af60304e389b4295adfafbfc85459962 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:16.108150 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2742c8aa-00cb-43ef-ad9d-8d0b53d8b925 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:16.223250 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2742c8aa-00cb-43ef-ad9d-8d0b53d8b925 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:16.274242 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cf6c26-f957-4ff5-aeea-c82919675fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:16.275399 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cf6c26-f957-4ff5-aeea-c82919675fe4 None admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:16.275680 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cf6c26-f957-4ff5-aeea-c82919675fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:16.275965 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cf6c26-f957-4ff5-aeea-c82919675fe4 None admin] PATH_INFO: `/v3/projects/af60304e389b4295adfafbfc85459962/groups/c1ed4738b21242e58797866b6c26ad9a/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:16.281765 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cf6c26-f957-4ff5-aeea-c82919675fe4 None admin] RBAC: Authorizing `identity:create_grant(project_id=af60304e389b4295adfafbfc85459962, group_id=c1ed4738b21242e58797866b6c26ad9a, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:16.283217 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cf6c26-f957-4ff5-aeea-c82919675fe4 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:16.348757 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 196/394] 10.222.0.42 () {58 vars in 1442 bytes} [Wed Jun 15 14:31:16 2022] PUT /identity/v3/projects/af60304e389b4295adfafbfc85459962/groups/c1ed4738b21242e58797866b6c26ad9a/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:31:16.633020 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28440ad3-2573-4e2c-a034-e1e05ef7b87f None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:16.633332 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28440ad3-2573-4e2c-a034-e1e05ef7b87f None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:16.633508 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28440ad3-2573-4e2c-a034-e1e05ef7b87f None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:16.635158 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 199/395] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:31:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:16.640748 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea642159-cc25-44e9-9c36-25680c43de48 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:16.641043 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea642159-cc25-44e9-9c36-25680c43de48 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:16.641325 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea642159-cc25-44e9-9c36-25680c43de48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:16.665154 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-ea642159-cc25-44e9-9c36-25680c43de48 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:16.666362 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-ea642159-cc25-44e9-9c36-25680c43de48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:16.684541 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:16.751137 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 197/396] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:31:16.775331 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28440ad3-2573-4e2c-a034-e1e05ef7b87f None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:16.888064 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:16.902229 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7a8307-c4c7-4735-b035-36febd3854da None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:16.902815 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7a8307-c4c7-4735-b035-36febd3854da None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:16.903261 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7a8307-c4c7-4735-b035-36febd3854da None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:16.905785 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 198/397] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:31:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:31:16.910915 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28440ad3-2573-4e2c-a034-e1e05ef7b87f None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:16.911473 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94bc680-727c-4256-a0b4-08ab76f28fe8 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:16.911962 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94bc680-727c-4256-a0b4-08ab76f28fe8 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:16.912487 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94bc680-727c-4256-a0b4-08ab76f28fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:16.932985 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-f94bc680-727c-4256-a0b4-08ab76f28fe8 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:16.934559 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-f94bc680-727c-4256-a0b4-08ab76f28fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:16.967078 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89efbfa3-a356-4f24-9dbd-4b9c15aaafc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:16.968033 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89efbfa3-a356-4f24-9dbd-4b9c15aaafc7 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:16.968273 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89efbfa3-a356-4f24-9dbd-4b9c15aaafc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:16.968566 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89efbfa3-a356-4f24-9dbd-4b9c15aaafc7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:16.999217 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89efbfa3-a356-4f24-9dbd-4b9c15aaafc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:31:17.007214 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-89efbfa3-a356-4f24-9dbd-4b9c15aaafc7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:31:17.007214 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:17.007214 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:17.007214 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:17.007214 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:17.007214 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:17.007214 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:17.007214 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:17.007214 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:17.007214 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:17.007214 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:17.007214 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:17.007214 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:31:17.007214 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:31:17.007214 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:31:17.007214 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:31:17.008160 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:31:17.008160 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:31:17.008160 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:31:17.008160 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:31:17.008160 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:17.008160 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:17.008160 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:17.008160 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:17.008160 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:17.008160 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:17.008160 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:17.008160 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:17.008160 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:17.008160 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:17.008160 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:17.008160 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:17.008960 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:17.008960 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:17.008960 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:17.008960 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:17.008960 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:31:17.008960 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:31:17.008960 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:31:17.008960 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:31:17.008960 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:31:17.008960 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:31:17.008960 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:31:17.008960 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:17.008960 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 200/398] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jun 15 14:31:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:17.015051 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89efbfa3-a356-4f24-9dbd-4b9c15aaafc7 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:17.044217 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 199/399] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:31:17.066470 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94bc680-727c-4256-a0b4-08ab76f28fe8 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:17.204560 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:17.223137 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89efbfa3-a356-4f24-9dbd-4b9c15aaafc7 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:17.253207 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94bc680-727c-4256-a0b4-08ab76f28fe8 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:17.266635 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01960b93-d5d9-4d34-a4bd-58a9c8888189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:17.267309 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01960b93-d5d9-4d34-a4bd-58a9c8888189 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:17.267589 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01960b93-d5d9-4d34-a4bd-58a9c8888189 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:17.267866 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01960b93-d5d9-4d34-a4bd-58a9c8888189 None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:17.268637 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01960b93-d5d9-4d34-a4bd-58a9c8888189 None admin] RBAC: Adding query filter params (name=member) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:17.268944 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01960b93-d5d9-4d34-a4bd-58a9c8888189 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:17.270324 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01960b93-d5d9-4d34-a4bd-58a9c8888189 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:17.287129 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 201/400] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Jun 15 14:31:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:17.292782 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01960b93-d5d9-4d34-a4bd-58a9c8888189 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:17.298473 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5f81da-620c-48e3-b405-db85f3ce5f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:17.299297 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f81da-620c-48e3-b405-db85f3ce5f81 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:17.299581 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f81da-620c-48e3-b405-db85f3ce5f81 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:17.299856 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f81da-620c-48e3-b405-db85f3ce5f81 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:17.306948 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:17.321345 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5f81da-620c-48e3-b405-db85f3ce5f81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:31:17.329922 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-8f5f81da-620c-48e3-b405-db85f3ce5f81 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:31:17.329922 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:17.329922 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:17.329922 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:17.329922 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:17.329922 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:17.329922 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:17.329922 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:17.329922 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:17.329922 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:17.329922 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:17.329922 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:17.329922 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:31:17.329922 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:31:17.329922 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:31:17.329922 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:31:17.330833 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:31:17.330833 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:31:17.330833 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:31:17.330833 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:31:17.330833 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:17.330833 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:17.330833 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:17.330833 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:17.330833 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:17.330833 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:17.330833 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:17.330833 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:17.330833 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:17.330833 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:17.330833 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:17.330833 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:17.331985 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:17.331985 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:17.331985 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:17.331985 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:17.331985 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:31:17.331985 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:31:17.331985 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:31:17.331985 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:31:17.331985 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:31:17.331985 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:31:17.331985 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:31:17.331985 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:17.331985 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 200/401] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Jun 15 14:31:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:31:17.339338 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5f81da-620c-48e3-b405-db85f3ce5f81 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:17.400172 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01960b93-d5d9-4d34-a4bd-58a9c8888189 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:17.468908 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70265f-b1f7-4b7e-bcb3-34bd1b001fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:17.469707 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70265f-b1f7-4b7e-bcb3-34bd1b001fa7 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:17.469990 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70265f-b1f7-4b7e-bcb3-34bd1b001fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:17.470268 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70265f-b1f7-4b7e-bcb3-34bd1b001fa7 None admin] PATH_INFO: `/v3/groups/c1ed4738b21242e58797866b6c26ad9a` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:17.471524 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f70265f-b1f7-4b7e-bcb3-34bd1b001fa7 None admin] RBAC: Authorizing `identity:get_group(group_id=c1ed4738b21242e58797866b6c26ad9a)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:17.473006 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f70265f-b1f7-4b7e-bcb3-34bd1b001fa7 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:17.474464 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5f81da-620c-48e3-b405-db85f3ce5f81 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:17.474942 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 202/402] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Jun 15 14:31:17 2022] GET /identity/v3/groups/c1ed4738b21242e58797866b6c26ad9a => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:31:17.480769 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70265f-b1f7-4b7e-bcb3-34bd1b001fa7 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:17.525645 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1507ea-516f-45c6-9908-7773e9b39ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:17.526487 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1507ea-516f-45c6-9908-7773e9b39ba8 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:17.526769 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1507ea-516f-45c6-9908-7773e9b39ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:17.527042 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1507ea-516f-45c6-9908-7773e9b39ba8 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:17.527789 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1507ea-516f-45c6-9908-7773e9b39ba8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:17.528109 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1507ea-516f-45c6-9908-7773e9b39ba8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:17.529594 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1507ea-516f-45c6-9908-7773e9b39ba8 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:17.546447 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:17.558107 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 201/403] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jun 15 14:31:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:31:17.564725 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1507ea-516f-45c6-9908-7773e9b39ba8 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:17.618533 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:17.656701 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70265f-b1f7-4b7e-bcb3-34bd1b001fa7 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:17.691276 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fa6b9a-c856-4e25-963c-85f8eadfcd63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:17.692158 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa6b9a-c856-4e25-963c-85f8eadfcd63 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:17.692449 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa6b9a-c856-4e25-963c-85f8eadfcd63 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:17.692728 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa6b9a-c856-4e25-963c-85f8eadfcd63 None admin] PATH_INFO: `/v3/projects/e58ea743785943a9936197cc15d05a80` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:17.693917 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fa6b9a-c856-4e25-963c-85f8eadfcd63 None admin] RBAC: Authorizing `identity:get_project(project_id=e58ea743785943a9936197cc15d05a80)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:17.695457 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fa6b9a-c856-4e25-963c-85f8eadfcd63 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:17.697485 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 203/404] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jun 15 14:31:17 2022] GET /identity/v3/projects/e58ea743785943a9936197cc15d05a80 => generated 302 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:31:17.698638 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff69904-a4fe-40aa-9436-a27582d19a9b None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:17.698922 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff69904-a4fe-40aa-9436-a27582d19a9b None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:17.699197 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff69904-a4fe-40aa-9436-a27582d19a9b None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:17.705147 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 204/405] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:31:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:17.705864 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff69904-a4fe-40aa-9436-a27582d19a9b None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:17.747564 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1507ea-516f-45c6-9908-7773e9b39ba8 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:17.789004 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f68771-8893-46f5-823b-c7f491d39511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:17.789860 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f68771-8893-46f5-823b-c7f491d39511 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:17.790563 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f68771-8893-46f5-823b-c7f491d39511 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:17.790861 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f68771-8893-46f5-823b-c7f491d39511 None admin] PATH_INFO: `/v3/groups/c1ed4738b21242e58797866b6c26ad9a` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:17.792065 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f68771-8893-46f5-823b-c7f491d39511 None admin] RBAC: Authorizing `identity:get_group(group_id=c1ed4738b21242e58797866b6c26ad9a)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:17.794317 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f68771-8893-46f5-823b-c7f491d39511 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:17.797178 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 202/406] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Jun 15 14:31:17 2022] GET /identity/v3/groups/c1ed4738b21242e58797866b6c26ad9a => generated 226 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:31:17.798642 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b748097-5a27-46cb-99ec-933668fdee1f None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:17.798929 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b748097-5a27-46cb-99ec-933668fdee1f None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:17.799203 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b748097-5a27-46cb-99ec-933668fdee1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:17.817683 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff69904-a4fe-40aa-9436-a27582d19a9b None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:17.821535 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-4b748097-5a27-46cb-99ec-933668fdee1f None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:17.822725 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-4b748097-5a27-46cb-99ec-933668fdee1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:17.867461 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88af3351-958f-4222-a04d-7ae54a2205ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:17.868458 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88af3351-958f-4222-a04d-7ae54a2205ee None admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:17.868740 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88af3351-958f-4222-a04d-7ae54a2205ee None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:17.869073 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88af3351-958f-4222-a04d-7ae54a2205ee None admin] PATH_INFO: `/v3/projects/e58ea743785943a9936197cc15d05a80/groups/c1ed4738b21242e58797866b6c26ad9a/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:17.871004 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88af3351-958f-4222-a04d-7ae54a2205ee None admin] RBAC: Authorizing `identity:create_grant(project_id=e58ea743785943a9936197cc15d05a80, group_id=c1ed4738b21242e58797866b6c26ad9a, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:17.874416 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88af3351-958f-4222-a04d-7ae54a2205ee None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:17.902039 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:17.908011 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 203/407] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:31:17.909031 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b748097-5a27-46cb-99ec-933668fdee1f None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:17.984662 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 205/408] 10.222.0.42 () {58 vars in 1442 bytes} [Wed Jun 15 14:31:17 2022] PUT /identity/v3/projects/e58ea743785943a9936197cc15d05a80/groups/c1ed4738b21242e58797866b6c26ad9a/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:31:17.985836 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88af3351-958f-4222-a04d-7ae54a2205ee None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:18.008888 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:18.068213 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b748097-5a27-46cb-99ec-933668fdee1f None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:18.147413 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d81eee0-1de6-4048-9520-e5cbce9d4d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:18.156109 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d81eee0-1de6-4048-9520-e5cbce9d4d6e None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:18.156109 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d81eee0-1de6-4048-9520-e5cbce9d4d6e None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:18.156109 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d81eee0-1de6-4048-9520-e5cbce9d4d6e None admin] PATH_INFO: `/v3/projects/e58ea743785943a9936197cc15d05a80` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:18.157841 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d81eee0-1de6-4048-9520-e5cbce9d4d6e None admin] RBAC: Authorizing `identity:get_project(project_id=e58ea743785943a9936197cc15d05a80)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:18.159099 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d81eee0-1de6-4048-9520-e5cbce9d4d6e None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:18.161479 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 204/409] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jun 15 14:31:17 2022] GET /identity/v3/projects/e58ea743785943a9936197cc15d05a80 => generated 302 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:18.167157 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d81eee0-1de6-4048-9520-e5cbce9d4d6e None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:18.195620 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88af3351-958f-4222-a04d-7ae54a2205ee None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:18.235392 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b2680d-7a88-44aa-8c86-a8c061bca616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:18.236384 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b2680d-7a88-44aa-8c86-a8c061bca616 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:18.236770 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b2680d-7a88-44aa-8c86-a8c061bca616 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:18.237235 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b2680d-7a88-44aa-8c86-a8c061bca616 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:18.248972 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:18.252419 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b2680d-7a88-44aa-8c86-a8c061bca616 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:31:18.267640 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-05b2680d-7a88-44aa-8c86-a8c061bca616 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:31:18.267640 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:18.267640 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:18.267640 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:18.267640 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:18.267640 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:18.267640 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:18.267640 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:18.267640 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:18.267640 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:18.267640 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:18.267640 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:18.267640 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:31:18.267640 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:31:18.267640 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:31:18.267640 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:31:18.268670 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:31:18.268670 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:31:18.268670 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:31:18.268670 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:31:18.268670 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:18.268670 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:18.268670 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:18.268670 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:18.268670 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:18.268670 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:18.268670 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:18.268670 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:18.268670 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:18.268670 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:18.268670 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:18.268670 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:18.269488 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:18.269488 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:18.269488 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:18.269488 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:18.269488 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:31:18.269488 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:31:18.269488 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:31:18.269488 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:31:18.269488 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:31:18.269488 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:31:18.269488 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:31:18.269488 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:18.270087 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 206/410] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jun 15 14:31:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:18.277032 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b2680d-7a88-44aa-8c86-a8c061bca616 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:18.311666 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d81eee0-1de6-4048-9520-e5cbce9d4d6e None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:18.381123 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ac9ff9-68fe-4201-9868-b0f667543590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:18.382010 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ac9ff9-68fe-4201-9868-b0f667543590 None admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:18.382288 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ac9ff9-68fe-4201-9868-b0f667543590 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:18.382563 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ac9ff9-68fe-4201-9868-b0f667543590 None admin] PATH_INFO: `/v3/projects/e58ea743785943a9936197cc15d05a80/groups/c1ed4738b21242e58797866b6c26ad9a/roles/61e98f694ee741a9894b7e6db6dd11d3` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:18.388779 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ac9ff9-68fe-4201-9868-b0f667543590 None admin] RBAC: Authorizing `identity:create_grant(project_id=e58ea743785943a9936197cc15d05a80, group_id=c1ed4738b21242e58797866b6c26ad9a, role_id=61e98f694ee741a9894b7e6db6dd11d3)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:18.390242 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ac9ff9-68fe-4201-9868-b0f667543590 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:18.407121 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:18.447476 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b2680d-7a88-44aa-8c86-a8c061bca616 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:18.476597 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 205/411] 10.222.0.42 () {58 vars in 1442 bytes} [Wed Jun 15 14:31:18 2022] PUT /identity/v3/projects/e58ea743785943a9936197cc15d05a80/groups/c1ed4738b21242e58797866b6c26ad9a/roles/61e98f694ee741a9894b7e6db6dd11d3 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:31:18.478512 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c216535-51cc-4d15-9c35-9a61fb4f8e43 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:18.483237 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c216535-51cc-4d15-9c35-9a61fb4f8e43 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:18.483588 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c216535-51cc-4d15-9c35-9a61fb4f8e43 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:18.489283 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f816c32-dd8e-46fb-8f91-5d2b9ff18e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:18.497800 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 206/412] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:31:18 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:18.497975 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f816c32-dd8e-46fb-8f91-5d2b9ff18e86 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:18.497975 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f816c32-dd8e-46fb-8f91-5d2b9ff18e86 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:18.497975 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f816c32-dd8e-46fb-8f91-5d2b9ff18e86 None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:18.497975 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f816c32-dd8e-46fb-8f91-5d2b9ff18e86 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:18.497975 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f816c32-dd8e-46fb-8f91-5d2b9ff18e86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:18.498786 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f816c32-dd8e-46fb-8f91-5d2b9ff18e86 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:18.502837 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220b3055-565b-4a95-a3da-2aecaf0dc24e None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:18.503131 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220b3055-565b-4a95-a3da-2aecaf0dc24e None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:18.503411 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220b3055-565b-4a95-a3da-2aecaf0dc24e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:18.510931 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 207/413] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jun 15 14:31:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:18.521485 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f816c32-dd8e-46fb-8f91-5d2b9ff18e86 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:18.544573 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-220b3055-565b-4a95-a3da-2aecaf0dc24e None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:18.546078 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-220b3055-565b-4a95-a3da-2aecaf0dc24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:18.651545 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 207/414] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:31:18.693262 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220b3055-565b-4a95-a3da-2aecaf0dc24e None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:18.702824 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f816c32-dd8e-46fb-8f91-5d2b9ff18e86 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:18.736346 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:18.750585 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dd6dab-2e86-4010-bd56-f655f0874113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:18.751754 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd6dab-2e86-4010-bd56-f655f0874113 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:18.754440 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd6dab-2e86-4010-bd56-f655f0874113 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:18.754440 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd6dab-2e86-4010-bd56-f655f0874113 None admin] PATH_INFO: `/v3/groups/ac0958f6024f49e2ab5ba11855c918bf` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:18.754794 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dd6dab-2e86-4010-bd56-f655f0874113 None admin] RBAC: Authorizing `identity:get_group(group_id=ac0958f6024f49e2ab5ba11855c918bf)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:18.756721 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dd6dab-2e86-4010-bd56-f655f0874113 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:18.759238 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 208/415] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Jun 15 14:31:18 2022] GET /identity/v3/groups/ac0958f6024f49e2ab5ba11855c918bf => generated 229 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:18.765779 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dd6dab-2e86-4010-bd56-f655f0874113 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:18.776139 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:18.866437 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220b3055-565b-4a95-a3da-2aecaf0dc24e None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:18.891984 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dd6dab-2e86-4010-bd56-f655f0874113 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:18.931173 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6221ca2b-1f6a-4a2d-905c-b25869e94a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:18.932038 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221ca2b-1f6a-4a2d-905c-b25869e94a40 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:18.932421 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221ca2b-1f6a-4a2d-905c-b25869e94a40 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:18.932702 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221ca2b-1f6a-4a2d-905c-b25869e94a40 None admin] PATH_INFO: `/v3/projects/688b80ae31764832aed9d6e0f3d3660f` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:18.933940 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6221ca2b-1f6a-4a2d-905c-b25869e94a40 None admin] RBAC: Authorizing `identity:get_project(project_id=688b80ae31764832aed9d6e0f3d3660f)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:18.935743 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6221ca2b-1f6a-4a2d-905c-b25869e94a40 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:18.937823 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 209/416] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jun 15 14:31:18 2022] GET /identity/v3/projects/688b80ae31764832aed9d6e0f3d3660f => generated 344 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:18.949132 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6221ca2b-1f6a-4a2d-905c-b25869e94a40 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:18.960605 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8c892b-ae9b-4035-9262-f6cf34614f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:18.961431 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8c892b-ae9b-4035-9262-f6cf34614f2d None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:18.961711 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8c892b-ae9b-4035-9262-f6cf34614f2d None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:18.961984 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8c892b-ae9b-4035-9262-f6cf34614f2d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:18.980491 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8c892b-ae9b-4035-9262-f6cf34614f2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:31:18.996770 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-be8c892b-ae9b-4035-9262-f6cf34614f2d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:31:18.996770 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:18.996770 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:18.996770 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:18.996770 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:18.996770 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:18.996770 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:18.996770 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:18.996770 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:18.996770 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:18.996770 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:18.996770 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:18.996770 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:31:18.996770 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:31:18.996770 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:31:18.996770 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:31:18.997827 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:31:18.997827 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:31:18.997827 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:31:18.997827 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:31:18.997827 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:18.997827 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:18.997827 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:18.997827 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:18.997827 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:18.997827 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:18.997827 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:18.997827 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:18.997827 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:18.997827 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:18.997827 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:18.997827 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:18.998737 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:18.998737 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:18.998737 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:18.998737 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:18.998737 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:31:18.998737 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:31:18.998737 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:31:18.998737 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:31:18.998737 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:31:18.998737 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:31:18.998737 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:31:18.998737 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:18.998737 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 208/417] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Jun 15 14:31:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:19.004275 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8c892b-ae9b-4035-9262-f6cf34614f2d None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:19.077833 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:19.156609 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6221ca2b-1f6a-4a2d-905c-b25869e94a40 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:19.205128 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a950a-3600-48f0-8f90-cc993929472e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:19.205329 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8c892b-ae9b-4035-9262-f6cf34614f2d None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:19.206008 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a950a-3600-48f0-8f90-cc993929472e None admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:19.206289 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a950a-3600-48f0-8f90-cc993929472e None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:19.206826 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a950a-3600-48f0-8f90-cc993929472e None admin] PATH_INFO: `/v3/projects/688b80ae31764832aed9d6e0f3d3660f/groups/ac0958f6024f49e2ab5ba11855c918bf/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:19.212321 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1a950a-3600-48f0-8f90-cc993929472e None admin] RBAC: Authorizing `identity:create_grant(project_id=688b80ae31764832aed9d6e0f3d3660f, group_id=ac0958f6024f49e2ab5ba11855c918bf, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:19.214488 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1a950a-3600-48f0-8f90-cc993929472e None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:19.222776 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:19.240414 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735e7036-b019-4831-bdd1-3f7ead80f9fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:19.241204 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e7036-b019-4831-bdd1-3f7ead80f9fe None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:19.241609 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e7036-b019-4831-bdd1-3f7ead80f9fe None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:19.241954 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e7036-b019-4831-bdd1-3f7ead80f9fe None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:19.242762 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735e7036-b019-4831-bdd1-3f7ead80f9fe None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:19.243181 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735e7036-b019-4831-bdd1-3f7ead80f9fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:19.244642 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735e7036-b019-4831-bdd1-3f7ead80f9fe None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:19.261048 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 209/418] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jun 15 14:31:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:19.261048 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeaa385-1334-48fc-9da0-cb9b38d16199 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:19.261048 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeaa385-1334-48fc-9da0-cb9b38d16199 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:19.261048 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeaa385-1334-48fc-9da0-cb9b38d16199 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:19.261048 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 210/419] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:31:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:19.266063 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeaa385-1334-48fc-9da0-cb9b38d16199 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:19.292584 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 210/420] 10.222.0.42 () {58 vars in 1442 bytes} [Wed Jun 15 14:31:18 2022] PUT /identity/v3/projects/688b80ae31764832aed9d6e0f3d3660f/groups/ac0958f6024f49e2ab5ba11855c918bf/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:31:19.302111 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b373ac36-c0a7-4094-9e35-3e18e4c561fa None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:19.302503 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b373ac36-c0a7-4094-9e35-3e18e4c561fa None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:19.302882 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b373ac36-c0a7-4094-9e35-3e18e4c561fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:19.333526 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-b373ac36-c0a7-4094-9e35-3e18e4c561fa None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:19.334910 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-b373ac36-c0a7-4094-9e35-3e18e4c561fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:19.464650 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 211/421] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:31:19.465359 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeaa385-1334-48fc-9da0-cb9b38d16199 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:19.491628 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b373ac36-c0a7-4094-9e35-3e18e4c561fa None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:19.515088 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565eae12-d2d1-402c-9b2a-cb5917cf829f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:19.515740 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565eae12-d2d1-402c-9b2a-cb5917cf829f None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:19.516020 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565eae12-d2d1-402c-9b2a-cb5917cf829f None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:19.516326 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565eae12-d2d1-402c-9b2a-cb5917cf829f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:19.557918 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:19.564954 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:19.590270 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565eae12-d2d1-402c-9b2a-cb5917cf829f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:19.592673 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565eae12-d2d1-402c-9b2a-cb5917cf829f None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:19.609553 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b373ac36-c0a7-4094-9e35-3e18e4c561fa None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:19.666146 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bc4b71-b88b-4e90-9cfc-9226269b04d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:19.667190 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc4b71-b88b-4e90-9cfc-9226269b04d5 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:19.667618 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc4b71-b88b-4e90-9cfc-9226269b04d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:19.668036 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc4b71-b88b-4e90-9cfc-9226269b04d5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:19.686632 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bc4b71-b88b-4e90-9cfc-9226269b04d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:31:19.694534 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 211/422] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jun 15 14:31:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:31:19.700975 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-f1bc4b71-b88b-4e90-9cfc-9226269b04d5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:31:19.700975 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:19.700975 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:19.700975 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:19.700975 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:19.700975 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:19.700975 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:19.700975 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:19.700975 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:19.700975 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:19.700975 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:19.700975 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:19.700975 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:31:19.700975 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:31:19.700975 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:31:19.700975 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:31:19.701987 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:31:19.701987 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:31:19.701987 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:31:19.701987 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:31:19.701987 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:19.701987 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:19.701987 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:19.701987 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:19.701987 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:19.701987 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:19.701987 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:19.701987 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:19.701987 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:19.701987 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:19.701987 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:19.701987 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:19.702986 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:19.702986 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:19.702986 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:19.702986 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:19.702986 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:31:19.702986 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:31:19.702986 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:31:19.702986 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:31:19.702986 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:31:19.702986 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:31:19.702986 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:31:19.702986 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:19.703732 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565eae12-d2d1-402c-9b2a-cb5917cf829f None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:19.705499 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 212/423] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jun 15 14:31:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:31:19.710563 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bc4b71-b88b-4e90-9cfc-9226269b04d5 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:19.837594 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:19.854769 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bc4b71-b88b-4e90-9cfc-9226269b04d5 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:19.870527 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:19.878551 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565eae12-d2d1-402c-9b2a-cb5917cf829f None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:19.904057 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60839c71-6546-437a-b531-41ca09eb44b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:19.905059 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60839c71-6546-437a-b531-41ca09eb44b4 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:19.905493 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60839c71-6546-437a-b531-41ca09eb44b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:19.905975 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60839c71-6546-437a-b531-41ca09eb44b4 None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:19.906987 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60839c71-6546-437a-b531-41ca09eb44b4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:19.907439 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60839c71-6546-437a-b531-41ca09eb44b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:19.909062 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60839c71-6546-437a-b531-41ca09eb44b4 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:19.924270 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3110895e-e363-4576-a106-cb9dbf09c1b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:19.925881 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3110895e-e363-4576-a106-cb9dbf09c1b6 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:19.926173 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3110895e-e363-4576-a106-cb9dbf09c1b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:19.926457 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3110895e-e363-4576-a106-cb9dbf09c1b6 None admin] PATH_INFO: `/v3/projects/af60304e389b4295adfafbfc85459962` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:19.927808 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3110895e-e363-4576-a106-cb9dbf09c1b6 None admin] RBAC: Authorizing `identity:get_project(project_id=af60304e389b4295adfafbfc85459962)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:19.929326 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3110895e-e363-4576-a106-cb9dbf09c1b6 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:19.929514 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 213/424] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Jun 15 14:31:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:31:19.931448 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 212/425] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jun 15 14:31:19 2022] GET /identity/v3/projects/af60304e389b4295adfafbfc85459962 => generated 298 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:19.937202 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60839c71-6546-437a-b531-41ca09eb44b4 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:19.947593 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3110895e-e363-4576-a106-cb9dbf09c1b6 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:20.052891 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3110895e-e363-4576-a106-cb9dbf09c1b6 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:20.114473 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d2a4b2-a25b-4cd4-8abc-b1d34ebb50e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:20.115264 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d2a4b2-a25b-4cd4-8abc-b1d34ebb50e1 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:20.115545 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d2a4b2-a25b-4cd4-8abc-b1d34ebb50e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:20.115821 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d2a4b2-a25b-4cd4-8abc-b1d34ebb50e1 None admin] PATH_INFO: `/v3/groups/c1ed4738b21242e58797866b6c26ad9a` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:20.117611 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d2a4b2-a25b-4cd4-8abc-b1d34ebb50e1 None admin] RBAC: Authorizing `identity:get_group(group_id=c1ed4738b21242e58797866b6c26ad9a)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:20.119040 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d2a4b2-a25b-4cd4-8abc-b1d34ebb50e1 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:20.126073 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 213/426] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Jun 15 14:31:19 2022] GET /identity/v3/groups/c1ed4738b21242e58797866b6c26ad9a => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:20.128901 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60839c71-6546-437a-b531-41ca09eb44b4 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:20.130839 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d2a4b2-a25b-4cd4-8abc-b1d34ebb50e1 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:20.160428 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:20.200287 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947d1bca-d178-448d-84d4-bf1283609cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:20.201018 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947d1bca-d178-448d-84d4-bf1283609cf1 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:20.201305 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947d1bca-d178-448d-84d4-bf1283609cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:20.201586 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947d1bca-d178-448d-84d4-bf1283609cf1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:20.283640 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:20.313070 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947d1bca-d178-448d-84d4-bf1283609cf1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:20.314956 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947d1bca-d178-448d-84d4-bf1283609cf1 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:20.338034 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d2a4b2-a25b-4cd4-8abc-b1d34ebb50e1 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:20.426371 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6350e1-062f-45ed-a14a-6a9f7165feee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:20.427437 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6350e1-062f-45ed-a14a-6a9f7165feee None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:20.427733 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6350e1-062f-45ed-a14a-6a9f7165feee None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:20.428020 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6350e1-062f-45ed-a14a-6a9f7165feee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:20.428919 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6350e1-062f-45ed-a14a-6a9f7165feee None admin] RBAC: Adding query filter params (group.id=c1ed4738b21242e58797866b6c26ad9a, role.id=61e98f694ee741a9894b7e6db6dd11d3, scope.project.id=af60304e389b4295adfafbfc85459962) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:20.429238 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6350e1-062f-45ed-a14a-6a9f7165feee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:20.430717 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6350e1-062f-45ed-a14a-6a9f7165feee None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:20.437119 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 214/427] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jun 15 14:31:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:31:20.442975 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947d1bca-d178-448d-84d4-bf1283609cf1 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:20.445388 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 214/428] 10.222.0.42 () {56 vars in 1376 bytes} [Wed Jun 15 14:31:20 2022] GET /identity/v3/role_assignments?group.id=c1ed4738b21242e58797866b6c26ad9a&scope.project.id=af60304e389b4295adfafbfc85459962&role.id=61e98f694ee741a9894b7e6db6dd11d3 => generated 611 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:20.509854 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b1be2a-6165-4dab-b5d7-a40800b6dccd None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:20.510213 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b1be2a-6165-4dab-b5d7-a40800b6dccd None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:20.510468 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b1be2a-6165-4dab-b5d7-a40800b6dccd None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:20.511918 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 215/429] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:31:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:31:20.523892 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c68902a-60a6-4a13-994f-ea0a9900245c None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:20.524237 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c68902a-60a6-4a13-994f-ea0a9900245c None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:20.524736 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c68902a-60a6-4a13-994f-ea0a9900245c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:20.571593 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:20.573296 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-2c68902a-60a6-4a13-994f-ea0a9900245c None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:20.574945 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-2c68902a-60a6-4a13-994f-ea0a9900245c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:20.601010 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947d1bca-d178-448d-84d4-bf1283609cf1 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:20.644617 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6252f1a-ef56-456a-9977-d9bbfb95400c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:20.645958 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6252f1a-ef56-456a-9977-d9bbfb95400c None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:20.646441 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6252f1a-ef56-456a-9977-d9bbfb95400c None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:20.646911 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6252f1a-ef56-456a-9977-d9bbfb95400c None admin] PATH_INFO: `/v3/projects/af60304e389b4295adfafbfc85459962` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:20.648452 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6252f1a-ef56-456a-9977-d9bbfb95400c None admin] RBAC: Authorizing `identity:get_project(project_id=af60304e389b4295adfafbfc85459962)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:20.651583 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6252f1a-ef56-456a-9977-d9bbfb95400c None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:20.654630 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 215/430] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jun 15 14:31:20 2022] GET /identity/v3/projects/af60304e389b4295adfafbfc85459962 => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:20.659446 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6252f1a-ef56-456a-9977-d9bbfb95400c None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:20.692923 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 216/431] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:31:20.719712 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c68902a-60a6-4a13-994f-ea0a9900245c None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:20.730297 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:20.786040 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6252f1a-ef56-456a-9977-d9bbfb95400c None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:20.831529 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426a37e4-d8e9-4b69-8f13-db67632286a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:20.833326 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a37e4-d8e9-4b69-8f13-db67632286a0 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:20.833821 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a37e4-d8e9-4b69-8f13-db67632286a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:20.834664 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a37e4-d8e9-4b69-8f13-db67632286a0 None admin] PATH_INFO: `/v3/groups/c1ed4738b21242e58797866b6c26ad9a` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:20.836398 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426a37e4-d8e9-4b69-8f13-db67632286a0 None admin] RBAC: Authorizing `identity:get_group(group_id=c1ed4738b21242e58797866b6c26ad9a)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:20.838998 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426a37e4-d8e9-4b69-8f13-db67632286a0 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:20.843041 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 216/432] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Jun 15 14:31:20 2022] GET /identity/v3/groups/c1ed4738b21242e58797866b6c26ad9a => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:20.848805 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426a37e4-d8e9-4b69-8f13-db67632286a0 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:20.862968 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c68902a-60a6-4a13-994f-ea0a9900245c None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:20.871770 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:20.912282 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df60eff4-b11b-4aa1-a598-248357bffcd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:20.914482 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df60eff4-b11b-4aa1-a598-248357bffcd5 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:20.915129 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df60eff4-b11b-4aa1-a598-248357bffcd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:20.915764 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df60eff4-b11b-4aa1-a598-248357bffcd5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:20.932279 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df60eff4-b11b-4aa1-a598-248357bffcd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:31:20.943086 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-df60eff4-b11b-4aa1-a598-248357bffcd5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:31:20.943086 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:20.943086 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:20.943086 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:20.943086 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:20.943086 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:20.943086 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:20.943086 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:20.943086 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:20.943086 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:20.943086 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:20.943086 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:20.943086 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:31:20.943086 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:31:20.943086 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:31:20.943086 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:31:20.943979 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:31:20.943979 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:31:20.943979 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:31:20.943979 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:31:20.943979 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:20.943979 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:20.943979 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:20.943979 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:20.943979 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:20.943979 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:20.943979 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:20.943979 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:20.943979 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:20.943979 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:20.943979 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:20.943979 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:20.944810 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:20.944810 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:20.944810 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:20.944810 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:20.944810 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:31:20.944810 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:31:20.944810 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:31:20.944810 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:31:20.944810 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:31:20.944810 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:31:20.944810 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:31:20.944810 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:20.946313 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 217/433] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jun 15 14:31:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:31:20.952292 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df60eff4-b11b-4aa1-a598-248357bffcd5 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:20.977734 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426a37e4-d8e9-4b69-8f13-db67632286a0 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:21.018951 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7790a2-f4de-44b5-87e7-afa26fde3617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:21.019387 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7790a2-f4de-44b5-87e7-afa26fde3617 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:21.019672 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7790a2-f4de-44b5-87e7-afa26fde3617 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:21.019950 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7790a2-f4de-44b5-87e7-afa26fde3617 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:21.022616 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7790a2-f4de-44b5-87e7-afa26fde3617 None admin] RBAC: Adding query filter params (group.id=c1ed4738b21242e58797866b6c26ad9a, role.id=9effaf496b874d01bd3f6d0555ef2d69, scope.project.id=af60304e389b4295adfafbfc85459962) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:21.022934 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7790a2-f4de-44b5-87e7-afa26fde3617 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:21.024742 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7790a2-f4de-44b5-87e7-afa26fde3617 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:21.032148 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 217/434] 10.222.0.42 () {56 vars in 1376 bytes} [Wed Jun 15 14:31:20 2022] GET /identity/v3/role_assignments?group.id=c1ed4738b21242e58797866b6c26ad9a&scope.project.id=af60304e389b4295adfafbfc85459962&role.id=9effaf496b874d01bd3f6d0555ef2d69 => generated 611 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:31:21.051459 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:21.089137 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df60eff4-b11b-4aa1-a598-248357bffcd5 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:21.133676 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec27488-875b-488f-9be0-94566c926688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:21.134381 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec27488-875b-488f-9be0-94566c926688 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:21.134670 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec27488-875b-488f-9be0-94566c926688 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:21.134948 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec27488-875b-488f-9be0-94566c926688 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:21.135760 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec27488-875b-488f-9be0-94566c926688 None admin] RBAC: Adding query filter params (name=member) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:21.136071 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec27488-875b-488f-9be0-94566c926688 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:21.137571 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec27488-875b-488f-9be0-94566c926688 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:21.148933 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 218/435] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Jun 15 14:31:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:31:21.155831 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7790a2-f4de-44b5-87e7-afa26fde3617 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:21.205340 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1894e651-28a3-4895-b09e-f29e8057fd3f None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:21.205660 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1894e651-28a3-4895-b09e-f29e8057fd3f None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:21.205838 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1894e651-28a3-4895-b09e-f29e8057fd3f None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:21.208159 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 219/436] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:31:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:31:21.213406 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26321037-5cde-420a-9d80-1fa6f778b283 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:21.213694 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26321037-5cde-420a-9d80-1fa6f778b283 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:21.213978 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26321037-5cde-420a-9d80-1fa6f778b283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:21.234161 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-26321037-5cde-420a-9d80-1fa6f778b283 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:21.235402 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-26321037-5cde-420a-9d80-1fa6f778b283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:21.259551 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:21.292301 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7790a2-f4de-44b5-87e7-afa26fde3617 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:21.328130 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 220/437] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:31:21.347852 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba125f2-5f86-4771-a339-eb0cd92603b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:21.348549 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba125f2-5f86-4771-a339-eb0cd92603b3 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:21.348834 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba125f2-5f86-4771-a339-eb0cd92603b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:21.349112 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba125f2-5f86-4771-a339-eb0cd92603b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:21.351513 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26321037-5cde-420a-9d80-1fa6f778b283 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:21.441184 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:21.442537 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba125f2-5f86-4771-a339-eb0cd92603b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:21.445960 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba125f2-5f86-4771-a339-eb0cd92603b3 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:21.493052 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26321037-5cde-420a-9d80-1fa6f778b283 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:21.547077 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:21.547989 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b79f6e-5eb2-48a3-9991-15168a19266a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:21.549392 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b79f6e-5eb2-48a3-9991-15168a19266a None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:21.550110 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b79f6e-5eb2-48a3-9991-15168a19266a None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:21.550523 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b79f6e-5eb2-48a3-9991-15168a19266a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:21.566324 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b79f6e-5eb2-48a3-9991-15168a19266a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:31:21.574754 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-97b79f6e-5eb2-48a3-9991-15168a19266a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:31:21.574754 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:21.574754 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:21.574754 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:21.574754 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:21.574754 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:21.574754 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:21.574754 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:21.574754 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:21.574754 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:21.574754 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:21.574754 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:21.574754 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:31:21.574754 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:31:21.574754 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:31:21.574754 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:31:21.575735 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:31:21.575735 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:31:21.575735 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:31:21.575735 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:31:21.575735 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:21.575735 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:21.575735 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:21.575735 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:21.575735 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:21.575735 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:21.575735 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:21.575735 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:21.575735 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:21.575735 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:21.575735 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:21.575735 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:21.577127 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:21.577127 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:21.577127 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:21.577127 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:21.577127 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:31:21.577127 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:31:21.577127 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:31:21.577127 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:31:21.577127 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:31:21.577127 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:31:21.577127 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:31:21.577127 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:21.577127 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 221/438] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Jun 15 14:31:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:31:21.583294 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 218/439] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jun 15 14:31:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:31:21.584638 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b79f6e-5eb2-48a3-9991-15168a19266a None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:21.586701 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba125f2-5f86-4771-a339-eb0cd92603b3 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:21.704540 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b79f6e-5eb2-48a3-9991-15168a19266a None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:21.715547 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba125f2-5f86-4771-a339-eb0cd92603b3 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:21.730018 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:21.748867 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a64fb6-ebf2-4091-9d05-47d3fe725c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:21.748867 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a64fb6-ebf2-4091-9d05-47d3fe725c0c None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:21.748867 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a64fb6-ebf2-4091-9d05-47d3fe725c0c None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:21.748867 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a64fb6-ebf2-4091-9d05-47d3fe725c0c None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:21.749252 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a64fb6-ebf2-4091-9d05-47d3fe725c0c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:21.749252 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a64fb6-ebf2-4091-9d05-47d3fe725c0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:21.750327 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a64fb6-ebf2-4091-9d05-47d3fe725c0c None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:21.762326 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 222/440] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jun 15 14:31:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:21.763920 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bbd80b-9c2c-410b-9cf4-dad5899865b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:21.764811 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bbd80b-9c2c-410b-9cf4-dad5899865b5 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:21.765095 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bbd80b-9c2c-410b-9cf4-dad5899865b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:21.765372 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bbd80b-9c2c-410b-9cf4-dad5899865b5 None admin] PATH_INFO: `/v3/projects/e58ea743785943a9936197cc15d05a80` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:21.766567 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bbd80b-9c2c-410b-9cf4-dad5899865b5 None admin] RBAC: Authorizing `identity:get_project(project_id=e58ea743785943a9936197cc15d05a80)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:21.768536 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bbd80b-9c2c-410b-9cf4-dad5899865b5 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:21.769730 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 219/441] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jun 15 14:31:21 2022] GET /identity/v3/projects/e58ea743785943a9936197cc15d05a80 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:21.770656 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a64fb6-ebf2-4091-9d05-47d3fe725c0c None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:21.775500 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bbd80b-9c2c-410b-9cf4-dad5899865b5 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:21.831549 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:21.884796 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a64fb6-ebf2-4091-9d05-47d3fe725c0c None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:21.885283 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bbd80b-9c2c-410b-9cf4-dad5899865b5 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:21.919473 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f464271-9bd0-4f1f-be0c-696eeac94a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:21.919869 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721f3061-321b-4d08-9f80-867aeb4420c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:21.920427 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f464271-9bd0-4f1f-be0c-696eeac94a80 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:21.920549 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f3061-321b-4d08-9f80-867aeb4420c5 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:21.920712 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f464271-9bd0-4f1f-be0c-696eeac94a80 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:21.920801 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f3061-321b-4d08-9f80-867aeb4420c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:21.920987 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f464271-9bd0-4f1f-be0c-696eeac94a80 None admin] PATH_INFO: `/v3/groups/c1ed4738b21242e58797866b6c26ad9a` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:21.921074 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f3061-321b-4d08-9f80-867aeb4420c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:21.922178 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f464271-9bd0-4f1f-be0c-696eeac94a80 None admin] RBAC: Authorizing `identity:get_group(group_id=c1ed4738b21242e58797866b6c26ad9a)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:21.923619 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f464271-9bd0-4f1f-be0c-696eeac94a80 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:21.925288 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 220/442] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Jun 15 14:31:21 2022] GET /identity/v3/groups/c1ed4738b21242e58797866b6c26ad9a => generated 226 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:31:21.926475 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ea31bf-81e6-415b-bd63-d232413f81c6 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:21.926763 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ea31bf-81e6-415b-bd63-d232413f81c6 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:21.927043 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ea31bf-81e6-415b-bd63-d232413f81c6 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:21.928516 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 221/443] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:31:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:21.932987 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ea31bf-81e6-415b-bd63-d232413f81c6 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:21.972601 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:21.991156 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721f3061-321b-4d08-9f80-867aeb4420c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:21.992674 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721f3061-321b-4d08-9f80-867aeb4420c5 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:22.034207 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ea31bf-81e6-415b-bd63-d232413f81c6 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:22.047045 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:22.067267 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fc2c5b-37c6-42d4-b659-87c3bc867832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:22.067913 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fc2c5b-37c6-42d4-b659-87c3bc867832 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:22.068233 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fc2c5b-37c6-42d4-b659-87c3bc867832 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:22.068510 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fc2c5b-37c6-42d4-b659-87c3bc867832 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:22.069331 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fc2c5b-37c6-42d4-b659-87c3bc867832 None admin] RBAC: Adding query filter params (group.id=c1ed4738b21242e58797866b6c26ad9a, role.id=9effaf496b874d01bd3f6d0555ef2d69, scope.project.id=e58ea743785943a9936197cc15d05a80) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:22.069642 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fc2c5b-37c6-42d4-b659-87c3bc867832 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:22.071056 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fc2c5b-37c6-42d4-b659-87c3bc867832 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:22.077840 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 222/444] 10.222.0.42 () {56 vars in 1376 bytes} [Wed Jun 15 14:31:21 2022] GET /identity/v3/role_assignments?group.id=c1ed4738b21242e58797866b6c26ad9a&scope.project.id=e58ea743785943a9936197cc15d05a80&role.id=9effaf496b874d01bd3f6d0555ef2d69 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:31:22.079258 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5d4db2-869a-4ef5-9d65-ea266c7d16c1 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:22.079825 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5d4db2-869a-4ef5-9d65-ea266c7d16c1 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:22.080129 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5d4db2-869a-4ef5-9d65-ea266c7d16c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:22.098103 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-be5d4db2-869a-4ef5-9d65-ea266c7d16c1 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:22.099352 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-be5d4db2-869a-4ef5-9d65-ea266c7d16c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:22.109664 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 223/445] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jun 15 14:31:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:31:22.115239 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721f3061-321b-4d08-9f80-867aeb4420c5 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:22.181909 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 223/446] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:31:22.206966 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5d4db2-869a-4ef5-9d65-ea266c7d16c1 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:22.214646 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:22.235749 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721f3061-321b-4d08-9f80-867aeb4420c5 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:22.273050 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c223965b-0eca-4e73-a79d-ff2c4b42b0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:22.273899 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c223965b-0eca-4e73-a79d-ff2c4b42b0ec None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:22.274174 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c223965b-0eca-4e73-a79d-ff2c4b42b0ec None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:22.274696 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c223965b-0eca-4e73-a79d-ff2c4b42b0ec None admin] PATH_INFO: `/v3/projects/e58ea743785943a9936197cc15d05a80` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:22.275868 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c223965b-0eca-4e73-a79d-ff2c4b42b0ec None admin] RBAC: Authorizing `identity:get_project(project_id=e58ea743785943a9936197cc15d05a80)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:22.277332 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c223965b-0eca-4e73-a79d-ff2c4b42b0ec None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:22.279586 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 224/447] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jun 15 14:31:22 2022] GET /identity/v3/projects/e58ea743785943a9936197cc15d05a80 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:31:22.285042 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c223965b-0eca-4e73-a79d-ff2c4b42b0ec None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:22.322216 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5d4db2-869a-4ef5-9d65-ea266c7d16c1 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:22.330329 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:22.362198 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b51e3c-c2cb-48ff-b144-f01587fd2115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:22.362999 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b51e3c-c2cb-48ff-b144-f01587fd2115 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:22.363282 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b51e3c-c2cb-48ff-b144-f01587fd2115 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:22.363558 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b51e3c-c2cb-48ff-b144-f01587fd2115 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:22.375298 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b51e3c-c2cb-48ff-b144-f01587fd2115 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:31:22.384381 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-30b51e3c-c2cb-48ff-b144-f01587fd2115 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:31:22.384381 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:22.384381 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:22.384381 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:22.384381 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:22.384381 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:22.384381 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:22.384381 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:22.384381 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:22.384381 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:22.384381 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:22.384381 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:22.384381 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:31:22.384381 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:31:22.384381 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:31:22.384381 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:31:22.385281 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:31:22.385281 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:31:22.385281 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:31:22.385281 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:31:22.385281 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:22.385281 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:22.385281 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:22.385281 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:22.385281 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:22.385281 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:22.385281 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:22.385281 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:22.385281 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:22.385281 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:22.385281 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:22.385281 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:22.386120 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:22.386120 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:22.386120 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:22.386120 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:22.386120 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:31:22.386120 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:31:22.386120 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:31:22.386120 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:31:22.386120 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:31:22.386120 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:31:22.386120 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:31:22.386120 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:22.386120 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 224/448] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jun 15 14:31:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:31:22.391428 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b51e3c-c2cb-48ff-b144-f01587fd2115 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:22.405702 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c223965b-0eca-4e73-a79d-ff2c4b42b0ec None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:22.442895 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852ce1b0-2043-491a-9d28-67ba5f7e74d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:22.443756 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ce1b0-2043-491a-9d28-67ba5f7e74d1 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:22.444044 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ce1b0-2043-491a-9d28-67ba5f7e74d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:22.444383 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ce1b0-2043-491a-9d28-67ba5f7e74d1 None admin] PATH_INFO: `/v3/groups/c1ed4738b21242e58797866b6c26ad9a` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:22.447445 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852ce1b0-2043-491a-9d28-67ba5f7e74d1 None admin] RBAC: Authorizing `identity:get_group(group_id=c1ed4738b21242e58797866b6c26ad9a)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:22.447445 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852ce1b0-2043-491a-9d28-67ba5f7e74d1 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:22.449846 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 225/449] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Jun 15 14:31:22 2022] GET /identity/v3/groups/c1ed4738b21242e58797866b6c26ad9a => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:31:22.457600 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852ce1b0-2043-491a-9d28-67ba5f7e74d1 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:22.477491 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:22.507953 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b51e3c-c2cb-48ff-b144-f01587fd2115 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:22.542319 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea9d13d-565f-4a6d-9b11-a2711aebc01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:22.542909 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea9d13d-565f-4a6d-9b11-a2711aebc01a None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:22.543193 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea9d13d-565f-4a6d-9b11-a2711aebc01a None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:22.543462 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea9d13d-565f-4a6d-9b11-a2711aebc01a None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:22.544296 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea9d13d-565f-4a6d-9b11-a2711aebc01a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:22.544606 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea9d13d-565f-4a6d-9b11-a2711aebc01a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:22.546188 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea9d13d-565f-4a6d-9b11-a2711aebc01a None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:22.554416 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 225/450] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jun 15 14:31:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:31:22.560140 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea9d13d-565f-4a6d-9b11-a2711aebc01a None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:22.566512 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852ce1b0-2043-491a-9d28-67ba5f7e74d1 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:22.588016 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:22.600963 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c87d016-a693-48f8-82a2-8d806b859af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:22.601614 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c87d016-a693-48f8-82a2-8d806b859af1 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:22.601900 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c87d016-a693-48f8-82a2-8d806b859af1 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:22.602180 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c87d016-a693-48f8-82a2-8d806b859af1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:22.603167 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c87d016-a693-48f8-82a2-8d806b859af1 None admin] RBAC: Adding query filter params (group.id=c1ed4738b21242e58797866b6c26ad9a, role.id=61e98f694ee741a9894b7e6db6dd11d3, scope.project.id=e58ea743785943a9936197cc15d05a80) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:22.603485 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c87d016-a693-48f8-82a2-8d806b859af1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:22.604938 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c87d016-a693-48f8-82a2-8d806b859af1 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:22.610954 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 226/451] 10.222.0.42 () {56 vars in 1376 bytes} [Wed Jun 15 14:31:22 2022] GET /identity/v3/role_assignments?group.id=c1ed4738b21242e58797866b6c26ad9a&scope.project.id=e58ea743785943a9936197cc15d05a80&role.id=61e98f694ee741a9894b7e6db6dd11d3 => generated 611 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:31:22.665528 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea9d13d-565f-4a6d-9b11-a2711aebc01a None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:22.702978 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2470aa26-3039-42db-8959-3d687843dfec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:22.703638 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2470aa26-3039-42db-8959-3d687843dfec None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:22.703917 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2470aa26-3039-42db-8959-3d687843dfec None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:22.704223 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2470aa26-3039-42db-8959-3d687843dfec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:22.774105 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2470aa26-3039-42db-8959-3d687843dfec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:22.775806 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2470aa26-3039-42db-8959-3d687843dfec None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:22.803349 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:22.876636 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 226/452] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jun 15 14:31:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 15 14:31:22.881200 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c87d016-a693-48f8-82a2-8d806b859af1 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:22.963216 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:22.982355 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c87d016-a693-48f8-82a2-8d806b859af1 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:23.015543 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ebab61-e612-4ffb-812a-3ebe3a04e348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:23.016413 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ebab61-e612-4ffb-812a-3ebe3a04e348 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:23.016693 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ebab61-e612-4ffb-812a-3ebe3a04e348 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:23.016966 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ebab61-e612-4ffb-812a-3ebe3a04e348 None admin] PATH_INFO: `/v3/projects/688b80ae31764832aed9d6e0f3d3660f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:23.018117 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ebab61-e612-4ffb-812a-3ebe3a04e348 None admin] RBAC: Authorizing `identity:get_project(project_id=688b80ae31764832aed9d6e0f3d3660f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:23.019542 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ebab61-e612-4ffb-812a-3ebe3a04e348 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:23.021237 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 227/453] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jun 15 14:31:22 2022] GET /identity/v3/projects/688b80ae31764832aed9d6e0f3d3660f => generated 344 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:23.026960 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2470aa26-3039-42db-8959-3d687843dfec None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:23.123315 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2470aa26-3039-42db-8959-3d687843dfec None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:23.148868 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:23.157195 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283c7f78-4509-41ed-86ce-afb7f0ef5df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:23.158025 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c7f78-4509-41ed-86ce-afb7f0ef5df1 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:23.158308 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c7f78-4509-41ed-86ce-afb7f0ef5df1 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:23.158582 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c7f78-4509-41ed-86ce-afb7f0ef5df1 None admin] PATH_INFO: `/v3/groups/ac0958f6024f49e2ab5ba11855c918bf` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:23.159737 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283c7f78-4509-41ed-86ce-afb7f0ef5df1 None admin] RBAC: Authorizing `identity:get_group(group_id=ac0958f6024f49e2ab5ba11855c918bf)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:23.161215 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283c7f78-4509-41ed-86ce-afb7f0ef5df1 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:23.162807 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 227/454] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Jun 15 14:31:23 2022] GET /identity/v3/groups/ac0958f6024f49e2ab5ba11855c918bf => generated 229 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:23.168600 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ebab61-e612-4ffb-812a-3ebe3a04e348 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:23.268031 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ebab61-e612-4ffb-812a-3ebe3a04e348 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:23.301177 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6860542a-f609-4334-a322-a516599070de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:23.302079 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6860542a-f609-4334-a322-a516599070de None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:23.302370 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6860542a-f609-4334-a322-a516599070de None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:23.302647 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6860542a-f609-4334-a322-a516599070de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:23.303472 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6860542a-f609-4334-a322-a516599070de None admin] RBAC: Adding query filter params (group.id=ac0958f6024f49e2ab5ba11855c918bf, role.id=2bd1979f53bb4853a50dd5bee6aa0782, scope.project.id=688b80ae31764832aed9d6e0f3d3660f) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:23.303787 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6860542a-f609-4334-a322-a516599070de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:23.305246 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6860542a-f609-4334-a322-a516599070de None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:23.311309 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 228/455] 10.222.0.42 () {56 vars in 1376 bytes} [Wed Jun 15 14:31:23 2022] GET /identity/v3/role_assignments?group.id=ac0958f6024f49e2ab5ba11855c918bf&scope.project.id=688b80ae31764832aed9d6e0f3d3660f&role.id=2bd1979f53bb4853a50dd5bee6aa0782 => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:25.970425 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedff635-f90b-44c2-a3d2-9b2574e88b67 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:25.970922 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedff635-f90b-44c2-a3d2-9b2574e88b67 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:25.970922 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedff635-f90b-44c2-a3d2-9b2574e88b67 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:25.972961 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 228/456] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:31:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:25.978819 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c91fa-4d1a-4ba8-bdea-3b4a64385681 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:25.979678 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c91fa-4d1a-4ba8-bdea-3b4a64385681 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:25.980162 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c91fa-4d1a-4ba8-bdea-3b4a64385681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:26.008662 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-b61c91fa-4d1a-4ba8-bdea-3b4a64385681 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:26.010336 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-b61c91fa-4d1a-4ba8-bdea-3b4a64385681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:26.025195 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:26.110486 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 229/457] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:31:26.123677 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63670eb-65ed-4a0e-b0b1-5df1360db589 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:26.124321 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63670eb-65ed-4a0e-b0b1-5df1360db589 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:26.124786 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63670eb-65ed-4a0e-b0b1-5df1360db589 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:26.126447 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 229/458] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:31:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:26.132492 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab475bde-e560-4d7c-9d54-6dc9ba4a2fb7 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:26.133007 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab475bde-e560-4d7c-9d54-6dc9ba4a2fb7 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:26.135092 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab475bde-e560-4d7c-9d54-6dc9ba4a2fb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:26.153126 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63670eb-65ed-4a0e-b0b1-5df1360db589 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:26.159225 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-ab475bde-e560-4d7c-9d54-6dc9ba4a2fb7 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:26.160507 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-ab475bde-e560-4d7c-9d54-6dc9ba4a2fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:26.282064 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 230/459] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:31:26.287159 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58242836-313b-4303-b0f2-1f6d3a7fd7be None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:26.287916 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58242836-313b-4303-b0f2-1f6d3a7fd7be None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:26.288559 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58242836-313b-4303-b0f2-1f6d3a7fd7be None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:26.290383 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 231/460] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:31:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:31:26.296194 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c108ff2-aacf-4496-b77b-0d2a9a732740 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:26.296729 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c108ff2-aacf-4496-b77b-0d2a9a732740 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:26.297235 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c108ff2-aacf-4496-b77b-0d2a9a732740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:26.324721 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-7c108ff2-aacf-4496-b77b-0d2a9a732740 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:26.326695 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-7c108ff2-aacf-4496-b77b-0d2a9a732740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:26.345266 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63670eb-65ed-4a0e-b0b1-5df1360db589 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:26.393028 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11258923-4238-47cd-9f5a-731febc63598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:26.394192 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11258923-4238-47cd-9f5a-731febc63598 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:26.394802 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11258923-4238-47cd-9f5a-731febc63598 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:26.395456 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11258923-4238-47cd-9f5a-731febc63598 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:26.407433 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11258923-4238-47cd-9f5a-731febc63598 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:26.409404 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11258923-4238-47cd-9f5a-731febc63598 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:26.414454 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 232/461] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:31:26.416036 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c108ff2-aacf-4496-b77b-0d2a9a732740 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:26.428742 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-11258923-4238-47cd-9f5a-731febc63598 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:26.428742 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:26.428742 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:26.428742 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:26.428742 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:26.428742 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:26.428742 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:26.428742 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:26.428742 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:26.428742 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:26.428742 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:26.428742 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:26.428742 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:31:26.428742 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:31:26.428742 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:31:26.429605 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:31:26.429605 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:26.429605 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:26.429605 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:26.429605 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:26.429605 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:26.429605 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:26.429605 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:26.429605 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:26.429605 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:26.429605 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:26.429605 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:26.429605 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:26.429605 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:26.429605 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:26.429605 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:26.429605 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:26.430459 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:31:26.430459 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:31:26.430459 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:26.430459 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:26.431404 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:26.433807 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 230/462] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:31:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:31:26.441978 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11258923-4238-47cd-9f5a-731febc63598 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:26.495901 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:26.557925 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11258923-4238-47cd-9f5a-731febc63598 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:26.571021 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c108ff2-aacf-4496-b77b-0d2a9a732740 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:26.601573 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf7685c-ad93-41ae-844a-c55a891f3864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:26.602360 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf7685c-ad93-41ae-844a-c55a891f3864 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:26.602652 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf7685c-ad93-41ae-844a-c55a891f3864 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:26.602972 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf7685c-ad93-41ae-844a-c55a891f3864 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:26.614474 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf7685c-ad93-41ae-844a-c55a891f3864 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:26.614619 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b7c5bb-d146-44f0-a6bb-3f603e422f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:26.615635 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b7c5bb-d146-44f0-a6bb-3f603e422f6c None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:26.615942 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf7685c-ad93-41ae-844a-c55a891f3864 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:26.616365 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b7c5bb-d146-44f0-a6bb-3f603e422f6c None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:26.616879 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b7c5bb-d146-44f0-a6bb-3f603e422f6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:26.628608 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b7c5bb-d146-44f0-a6bb-3f603e422f6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:26.630342 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b7c5bb-d146-44f0-a6bb-3f603e422f6c None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:26.641293 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-abf7685c-ad93-41ae-844a-c55a891f3864 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:26.641293 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:26.641293 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:26.641293 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:26.641293 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:26.641293 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:26.641293 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:26.641293 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:26.641293 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:26.641293 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:26.641293 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:26.641293 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:26.641293 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:31:26.641293 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:31:26.641293 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:31:26.642225 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:31:26.642225 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:26.642225 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:26.642225 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:26.642225 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:26.642225 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:26.642225 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:26.642225 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:26.642225 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:26.642225 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:26.642225 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:26.642225 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:26.642225 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:26.642225 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:26.642225 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:26.642225 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:26.642225 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:26.643068 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:31:26.643068 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:31:26.643068 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:26.643068 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:26.648381 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 231/463] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:31:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:26.648381 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf7685c-ad93-41ae-844a-c55a891f3864 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:26.657975 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-e6b7c5bb-d146-44f0-a6bb-3f603e422f6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:26.657975 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:26.657975 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:26.657975 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:26.657975 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:26.657975 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:26.657975 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:26.657975 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:26.657975 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:26.657975 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:26.657975 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:26.657975 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:26.657975 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:31:26.657975 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:31:26.657975 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:31:26.658825 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:31:26.658825 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:26.658825 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:26.658825 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:26.658825 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:26.658825 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:26.658825 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:26.658825 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:26.658825 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:26.658825 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:26.658825 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:26.658825 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:26.658825 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:26.658825 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:26.658825 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:26.658825 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:26.658825 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:26.659711 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:31:26.659711 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:31:26.659711 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:26.659711 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:26.661990 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 233/464] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:31:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:26.663080 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b7c5bb-d146-44f0-a6bb-3f603e422f6c None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:26.742193 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:26.757825 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:26.760996 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf7685c-ad93-41ae-844a-c55a891f3864 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:26.775414 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b7c5bb-d146-44f0-a6bb-3f603e422f6c None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:26.800618 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810045d2-c3cc-422b-975f-56cf98bd69ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:26.801315 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810045d2-c3cc-422b-975f-56cf98bd69ac None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:26.801615 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810045d2-c3cc-422b-975f-56cf98bd69ac None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:26.801895 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810045d2-c3cc-422b-975f-56cf98bd69ac None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:26.802589 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810045d2-c3cc-422b-975f-56cf98bd69ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:26.802950 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810045d2-c3cc-422b-975f-56cf98bd69ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:26.804391 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810045d2-c3cc-422b-975f-56cf98bd69ac None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:26.817272 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7256871c-51fd-4cc4-afe9-328076b086c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:26.818261 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7256871c-51fd-4cc4-afe9-328076b086c3 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:26.818738 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7256871c-51fd-4cc4-afe9-328076b086c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:26.819203 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7256871c-51fd-4cc4-afe9-328076b086c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:26.820192 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7256871c-51fd-4cc4-afe9-328076b086c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:26.820694 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7256871c-51fd-4cc4-afe9-328076b086c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:26.822265 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7256871c-51fd-4cc4-afe9-328076b086c3 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:26.822745 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 232/465] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:31:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:26.823570 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810045d2-c3cc-422b-975f-56cf98bd69ac None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:26.836367 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 234/466] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:31:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:26.837550 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7256871c-51fd-4cc4-afe9-328076b086c3 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:26.939143 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810045d2-c3cc-422b-975f-56cf98bd69ac None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:26.954476 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7256871c-51fd-4cc4-afe9-328076b086c3 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:26.973391 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:26.983544 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f32ecf0-0738-4dcf-8120-17e8ef489df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:26.984251 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32ecf0-0738-4dcf-8120-17e8ef489df4 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:26.984525 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32ecf0-0738-4dcf-8120-17e8ef489df4 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:26.984797 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32ecf0-0738-4dcf-8120-17e8ef489df4 None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:26.985569 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f32ecf0-0738-4dcf-8120-17e8ef489df4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:26.985880 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f32ecf0-0738-4dcf-8120-17e8ef489df4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:26.987253 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f32ecf0-0738-4dcf-8120-17e8ef489df4 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:26.996640 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32663c41-979b-42c5-ba00-409b4e6328b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:26.997893 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32663c41-979b-42c5-ba00-409b4e6328b8 None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:26.998181 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32663c41-979b-42c5-ba00-409b4e6328b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:26.998440 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32663c41-979b-42c5-ba00-409b4e6328b8 None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:26.999141 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32663c41-979b-42c5-ba00-409b4e6328b8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:26.999997 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 233/467] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:31:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:27.000598 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32663c41-979b-42c5-ba00-409b4e6328b8 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:27.005007 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f32ecf0-0738-4dcf-8120-17e8ef489df4 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:27.027359 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:27.103325 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f32ecf0-0738-4dcf-8120-17e8ef489df4 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:27.116273 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 235/468] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Jun 15 14:31:26 2022] POST /identity/v3/users => generated 246 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:31:27.116273 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1939e1-3b5d-482e-bd00-2d8e9680be47 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:27.116273 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1939e1-3b5d-482e-bd00-2d8e9680be47 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:27.116273 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1939e1-3b5d-482e-bd00-2d8e9680be47 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:27.116273 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 236/469] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:31:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:27.116273 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1939e1-3b5d-482e-bd00-2d8e9680be47 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:27.157704 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aabab4-bb26-4e45-9f95-766c222f42a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:27.158578 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aabab4-bb26-4e45-9f95-766c222f42a0 None admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:27.158863 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aabab4-bb26-4e45-9f95-766c222f42a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:27.159141 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aabab4-bb26-4e45-9f95-766c222f42a0 None admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:27.159866 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aabab4-bb26-4e45-9f95-766c222f42a0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:27.162487 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aabab4-bb26-4e45-9f95-766c222f42a0 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:27.244743 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 234/470] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Jun 15 14:31:27 2022] POST /identity/v3/users => generated 248 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:31:27.246254 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1939e1-3b5d-482e-bd00-2d8e9680be47 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:27.246569 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b608e2a-1402-4527-b69f-5af2c7c1e41d None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:27.247524 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b608e2a-1402-4527-b69f-5af2c7c1e41d None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:27.247864 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b608e2a-1402-4527-b69f-5af2c7c1e41d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:27.283087 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-0b608e2a-1402-4527-b69f-5af2c7c1e41d None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:27.284635 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-0b608e2a-1402-4527-b69f-5af2c7c1e41d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:27.297005 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1a69d2-4e0f-4239-9e9f-1ee9e9fe7d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:27.298797 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a69d2-4e0f-4239-9e9f-1ee9e9fe7d61 None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:27.299089 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a69d2-4e0f-4239-9e9f-1ee9e9fe7d61 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:27.299406 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a69d2-4e0f-4239-9e9f-1ee9e9fe7d61 None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:27.300110 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1a69d2-4e0f-4239-9e9f-1ee9e9fe7d61 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:27.301620 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1a69d2-4e0f-4239-9e9f-1ee9e9fe7d61 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:27.394907 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 235/471] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:31:27.423225 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 237/472] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Jun 15 14:31:27 2022] POST /identity/v3/users => generated 248 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:31:27.426234 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b608e2a-1402-4527-b69f-5af2c7c1e41d None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:27.436821 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:27.461406 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4f3157-763f-458d-8608-4a1c6e9c9478 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:27.461700 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4f3157-763f-458d-8608-4a1c6e9c9478 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:27.461997 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4f3157-763f-458d-8608-4a1c6e9c9478 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:27.463765 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 238/473] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:31:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:27.473074 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94072f2c-c93a-473b-a05f-726cbdd26519 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:27.473473 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94072f2c-c93a-473b-a05f-726cbdd26519 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:27.473762 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94072f2c-c93a-473b-a05f-726cbdd26519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:27.502173 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-94072f2c-c93a-473b-a05f-726cbdd26519 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:27.504320 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-94072f2c-c93a-473b-a05f-726cbdd26519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:27.546283 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:27.571931 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b608e2a-1402-4527-b69f-5af2c7c1e41d None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:27.612246 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 239/474] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:31:27.619193 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2582f428-919b-4dcd-965a-3ef9b22628ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:27.620369 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2582f428-919b-4dcd-965a-3ef9b22628ab None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:27.620796 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2582f428-919b-4dcd-965a-3ef9b22628ab None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:27.621212 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2582f428-919b-4dcd-965a-3ef9b22628ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:27.634408 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2582f428-919b-4dcd-965a-3ef9b22628ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:27.635110 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94072f2c-c93a-473b-a05f-726cbdd26519 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:27.636313 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2582f428-919b-4dcd-965a-3ef9b22628ab None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:27.657427 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-2582f428-919b-4dcd-965a-3ef9b22628ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:27.657427 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:27.657427 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:27.657427 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:27.657427 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:27.657427 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:27.657427 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:27.657427 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:27.657427 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:27.657427 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:27.657427 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:27.657427 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:27.657427 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:31:27.657427 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:31:27.657427 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:31:27.658289 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:31:27.658289 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:27.658289 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:27.658289 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:27.658289 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:27.658289 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:27.658289 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:27.658289 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:27.658289 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:27.658289 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:27.658289 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:27.658289 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:27.658289 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:27.658289 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:27.658289 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:27.658289 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:27.658289 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:27.659132 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:31:27.659132 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:31:27.659132 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:27.659132 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:27.659132 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 236/475] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:31:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:27.664651 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2582f428-919b-4dcd-965a-3ef9b22628ab None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:27.748056 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:27.776433 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94072f2c-c93a-473b-a05f-726cbdd26519 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:27.803695 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2582f428-919b-4dcd-965a-3ef9b22628ab None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:27.823372 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3630c45b-831e-4cc0-9eb9-480ccc66b3d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:27.824368 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3630c45b-831e-4cc0-9eb9-480ccc66b3d4 None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:27.824829 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3630c45b-831e-4cc0-9eb9-480ccc66b3d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:27.825830 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3630c45b-831e-4cc0-9eb9-480ccc66b3d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:27.826915 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3630c45b-831e-4cc0-9eb9-480ccc66b3d4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:27.828908 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3630c45b-831e-4cc0-9eb9-480ccc66b3d4 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:27.843475 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b8717f-0d1f-48a3-afb2-0fd340109aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:27.844186 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b8717f-0d1f-48a3-afb2-0fd340109aac None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:27.844476 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b8717f-0d1f-48a3-afb2-0fd340109aac None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:27.844755 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b8717f-0d1f-48a3-afb2-0fd340109aac None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:27.845490 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b8717f-0d1f-48a3-afb2-0fd340109aac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:27.845802 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b8717f-0d1f-48a3-afb2-0fd340109aac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:27.847451 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b8717f-0d1f-48a3-afb2-0fd340109aac None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:27.860942 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 237/476] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:31:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:31:27.866417 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b8717f-0d1f-48a3-afb2-0fd340109aac None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:27.878960 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.sql.core [None req-3630c45b-831e-4cc0-9eb9-480ccc66b3d4 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 15 14:31:27.878960 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 15 14:31:27.878960 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [parameters: {'id': 'ecce8976aedd44b8a8a5205cf1d48118', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 15 14:31:27.878960 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80600) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 15 14:31:27.911240 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-3630c45b-831e-4cc0-9eb9-480ccc66b3d4 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 15 14:31:27.911240 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:27.911240 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 15 14:31:27.911240 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 15 14:31:27.911240 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 15 14:31:27.911240 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 15 14:31:27.911240 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 15 14:31:27.911240 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application result = self._query(query) Jun 15 14:31:27.911240 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 15 14:31:27.911240 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application conn.query(q) Jun 15 14:31:27.911240 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 15 14:31:27.911240 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 15 14:31:27.911240 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 15 14:31:27.911240 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application result.read() Jun 15 14:31:27.911240 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 15 14:31:27.912164 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 15 14:31:27.912164 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 15 14:31:27.912164 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 15 14:31:27.912164 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 15 14:31:27.912164 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 15 14:31:27.912164 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 15 14:31:27.912164 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 15 14:31:27.912164 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 15 14:31:27.912164 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:27.912164 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 15 14:31:27.912164 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:27.912164 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:27.912164 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 15 14:31:27.912164 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 15 14:31:27.912164 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 15 14:31:27.912164 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return ref.to_dict() Jun 15 14:31:27.912164 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 15 14:31:27.912164 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application next(self.gen) Jun 15 14:31:27.913109 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 15 14:31:27.913109 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application yield resource Jun 15 14:31:27.913109 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 15 14:31:27.913109 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application next(self.gen) Jun 15 14:31:27.913109 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 15 14:31:27.913109 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self.session.rollback() Jun 15 14:31:27.913109 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 15 14:31:27.913109 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self.force_reraise() Jun 15 14:31:27.913109 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 15 14:31:27.913109 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise self.value Jun 15 14:31:27.913109 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 15 14:31:27.913109 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 15 14:31:27.913109 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 15 14:31:27.913109 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application session.commit() Jun 15 14:31:27.913109 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 15 14:31:27.913109 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 15 14:31:27.913109 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 15 14:31:27.914002 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._prepare_impl() Jun 15 14:31:27.914002 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 15 14:31:27.914002 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self.session.flush() Jun 15 14:31:27.914002 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 15 14:31:27.914002 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._flush(objects) Jun 15 14:31:27.914002 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 15 14:31:27.914002 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 15 14:31:27.914002 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 15 14:31:27.914002 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application compat.raise_( Jun 15 14:31:27.914002 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 15 14:31:27.914002 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception Jun 15 14:31:27.914002 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 15 14:31:27.914002 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application flush_context.execute() Jun 15 14:31:27.914002 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 15 14:31:27.914002 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rec.execute(self) Jun 15 14:31:27.914002 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 15 14:31:27.914002 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 15 14:31:27.914870 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 15 14:31:27.914870 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 15 14:31:27.914870 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 15 14:31:27.914870 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 15 14:31:27.914870 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 15 14:31:27.914870 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 15 14:31:27.914870 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 15 14:31:27.914870 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 15 14:31:27.914870 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 15 14:31:27.914870 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 15 14:31:27.914870 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 15 14:31:27.914870 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 15 14:31:27.914870 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 15 14:31:27.914870 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 15 14:31:27.914870 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 15 14:31:27.914870 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception Jun 15 14:31:27.915678 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 15 14:31:27.915678 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 15 14:31:27.915678 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 15 14:31:27.915678 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 15 14:31:27.915678 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 15 14:31:27.915678 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application result = self._query(query) Jun 15 14:31:27.915678 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 15 14:31:27.915678 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application conn.query(q) Jun 15 14:31:27.915678 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 15 14:31:27.915678 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 15 14:31:27.915678 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 15 14:31:27.915678 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application result.read() Jun 15 14:31:27.915678 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 15 14:31:27.915678 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 15 14:31:27.915678 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 15 14:31:27.915678 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 15 14:31:27.918519 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 15 14:31:27.918519 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 15 14:31:27.918519 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 15 14:31:27.918519 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 15 14:31:27.918519 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 15 14:31:27.918519 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 15 14:31:27.918519 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [parameters: {'id': 'ecce8976aedd44b8a8a5205cf1d48118', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 15 14:31:27.918519 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 15 14:31:27.918519 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:27.918519 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:31:27.918519 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:27.918519 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:27.918519 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:27.918519 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:27.918519 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:27.918519 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:27.921167 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:27.921167 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:27.921167 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:27.921167 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:27.921167 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:27.921167 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:27.921167 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 15 14:31:27.921167 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 15 14:31:27.921167 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:27.921167 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:27.921167 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 15 14:31:27.921167 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 15 14:31:27.921167 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 15 14:31:27.921167 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 15 14:31:27.921167 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 15 14:31:27.921167 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:27.929705 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 240/477] 10.222.0.42 () {60 vars in 1131 bytes} [Wed Jun 15 14:31:27 2022] POST /identity/v3/roles => generated 144 bytes in 278 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:31:27.929705 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3630c45b-831e-4cc0-9eb9-480ccc66b3d4 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:27.982420 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:28.043117 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b8717f-0d1f-48a3-afb2-0fd340109aac None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:28.083060 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:28.113733 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13309e84-3e15-4c5b-9747-9771cffb2228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:28.114416 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13309e84-3e15-4c5b-9747-9771cffb2228 None admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:28.114700 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13309e84-3e15-4c5b-9747-9771cffb2228 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:28.114973 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13309e84-3e15-4c5b-9747-9771cffb2228 None admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:28.115693 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13309e84-3e15-4c5b-9747-9771cffb2228 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:28.117172 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13309e84-3e15-4c5b-9747-9771cffb2228 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:28.140781 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3630c45b-831e-4cc0-9eb9-480ccc66b3d4 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:28.181105 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05841ba-7bef-4a04-81a6-5c3fcd85463f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:28.181938 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05841ba-7bef-4a04-81a6-5c3fcd85463f None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:28.182236 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05841ba-7bef-4a04-81a6-5c3fcd85463f None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:28.182558 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05841ba-7bef-4a04-81a6-5c3fcd85463f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:28.194544 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05841ba-7bef-4a04-81a6-5c3fcd85463f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:31:28.203888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-c05841ba-7bef-4a04-81a6-5c3fcd85463f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:31:28.203888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:28.203888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:28.203888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:28.203888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:28.203888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:28.203888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:28.203888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:28.203888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:28.203888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:28.203888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:28.203888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:28.203888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:31:28.203888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:31:28.203888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:31:28.203888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:31:28.204814 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:31:28.204814 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:31:28.204814 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:31:28.204814 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:31:28.204814 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:28.204814 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:28.204814 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:28.204814 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:28.204814 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:28.204814 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:28.204814 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:28.204814 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:28.204814 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:28.204814 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:28.204814 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:28.204814 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:28.205612 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:28.205612 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:28.205612 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:28.205612 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:28.205612 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:31:28.205612 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:31:28.205612 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:31:28.205612 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:31:28.205612 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:31:28.205612 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:31:28.205612 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:31:28.205612 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:28.209789 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 241/478] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Jun 15 14:31:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:31:28.210949 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05841ba-7bef-4a04-81a6-5c3fcd85463f None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:28.239539 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 238/479] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Jun 15 14:31:27 2022] POST /identity/v3/users => generated 249 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:31:28.306221 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:28.326748 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05841ba-7bef-4a04-81a6-5c3fcd85463f None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:28.364514 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ef4d02-40ad-44d0-903d-ea946ad61a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:28.365407 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef4d02-40ad-44d0-903d-ea946ad61a81 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:28.365935 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef4d02-40ad-44d0-903d-ea946ad61a81 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:28.366617 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef4d02-40ad-44d0-903d-ea946ad61a81 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:28.367445 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ef4d02-40ad-44d0-903d-ea946ad61a81 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:28.367760 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ef4d02-40ad-44d0-903d-ea946ad61a81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:28.369183 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ef4d02-40ad-44d0-903d-ea946ad61a81 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:28.382889 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 242/480] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jun 15 14:31:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:29.668880 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6009870-a7d9-4dd7-bd71-44368d75488d None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:29.669385 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6009870-a7d9-4dd7-bd71-44368d75488d None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:29.669466 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6009870-a7d9-4dd7-bd71-44368d75488d None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:29.671081 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 239/481] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:31:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:29.676024 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47d347-f327-4586-8b67-4c3c2a5a2ede None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:29.676430 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47d347-f327-4586-8b67-4c3c2a5a2ede None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:29.676612 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47d347-f327-4586-8b67-4c3c2a5a2ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:29.709319 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-6f47d347-f327-4586-8b67-4c3c2a5a2ede None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:29.710507 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-6f47d347-f327-4586-8b67-4c3c2a5a2ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:29.804776 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 243/482] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:31:29.844054 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6009870-a7d9-4dd7-bd71-44368d75488d None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:29.992923 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6009870-a7d9-4dd7-bd71-44368d75488d None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:30.085433 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4d0b54-0923-4a84-a9a1-19f4f05a741d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:30.086148 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d0b54-0923-4a84-a9a1-19f4f05a741d None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:30.086431 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d0b54-0923-4a84-a9a1-19f4f05a741d None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:30.086711 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d0b54-0923-4a84-a9a1-19f4f05a741d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:30.102442 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4d0b54-0923-4a84-a9a1-19f4f05a741d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:31:30.110319 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-1a4d0b54-0923-4a84-a9a1-19f4f05a741d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:31:30.110319 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:30.110319 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:30.110319 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:30.110319 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:30.110319 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:30.110319 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:30.110319 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:30.110319 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:30.110319 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:30.110319 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:30.110319 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:30.110319 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:31:30.110319 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:31:30.110319 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:31:30.110319 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:31:30.111199 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:31:30.111199 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:31:30.111199 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:31:30.111199 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:31:30.111199 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:30.111199 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:30.111199 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:30.111199 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:30.111199 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:30.111199 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:30.111199 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:30.111199 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:30.111199 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:30.111199 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:30.111199 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:30.111199 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:30.111979 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:30.111979 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:30.111979 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:30.111979 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:30.111979 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:31:30.111979 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:31:30.111979 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:31:30.111979 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:31:30.111979 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:31:30.111979 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:31:30.111979 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:31:30.111979 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:30.111979 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 240/483] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Jun 15 14:31:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:30.117470 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47d347-f327-4586-8b67-4c3c2a5a2ede None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:30.139453 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd74e65-adcd-46c6-b742-b2e535774e78 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:30.139744 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd74e65-adcd-46c6-b742-b2e535774e78 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:30.140012 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd74e65-adcd-46c6-b742-b2e535774e78 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:30.142193 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 241/484] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:31:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:30.171855 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ab6962-acbc-498a-883c-96872bec9816 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:30.172177 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ab6962-acbc-498a-883c-96872bec9816 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:30.172448 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ab6962-acbc-498a-883c-96872bec9816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:30.197074 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-f4ab6962-acbc-498a-883c-96872bec9816 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:30.198273 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-f4ab6962-acbc-498a-883c-96872bec9816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:30.262038 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:30.268894 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:30.304781 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47d347-f327-4586-8b67-4c3c2a5a2ede None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:30.317213 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 242/485] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:31:30.325907 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b073fd28-143b-4133-a46c-bc7a6129314b None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:30.326161 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b073fd28-143b-4133-a46c-bc7a6129314b None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:30.326241 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b073fd28-143b-4133-a46c-bc7a6129314b None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:30.327585 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 243/486] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:31:30 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:30.337437 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452ae44e-c183-49a1-9f7d-189551510b4e None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:30.337437 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452ae44e-c183-49a1-9f7d-189551510b4e None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:30.337437 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452ae44e-c183-49a1-9f7d-189551510b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:30.355153 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42872af4-aa29-4381-b67f-b07d22f74d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:30.356300 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42872af4-aa29-4381-b67f-b07d22f74d65 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:30.357021 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-452ae44e-c183-49a1-9f7d-189551510b4e None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:30.357468 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42872af4-aa29-4381-b67f-b07d22f74d65 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:30.357961 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42872af4-aa29-4381-b67f-b07d22f74d65 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:30.358463 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-452ae44e-c183-49a1-9f7d-189551510b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:30.359135 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42872af4-aa29-4381-b67f-b07d22f74d65 None admin] RBAC: Adding query filter params (name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:30.359641 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42872af4-aa29-4381-b67f-b07d22f74d65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:30.361260 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42872af4-aa29-4381-b67f-b07d22f74d65 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:30.370453 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 244/487] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Jun 15 14:31:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:30.371701 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42872af4-aa29-4381-b67f-b07d22f74d65 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:30.436118 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 244/488] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:31:30.436118 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452ae44e-c183-49a1-9f7d-189551510b4e None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:30.504237 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42872af4-aa29-4381-b67f-b07d22f74d65 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:30.528717 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:30.534726 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452ae44e-c183-49a1-9f7d-189551510b4e None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:30.547491 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983d29fd-c33a-4c00-b50c-2193df78eb06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:30.553607 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983d29fd-c33a-4c00-b50c-2193df78eb06 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:30.553999 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983d29fd-c33a-4c00-b50c-2193df78eb06 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:30.554379 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983d29fd-c33a-4c00-b50c-2193df78eb06 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:30.557645 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:30.567911 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983d29fd-c33a-4c00-b50c-2193df78eb06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:31:30.575076 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e95826d-3bda-470a-8ffc-cb3c1ec162a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:30.576818 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95826d-3bda-470a-8ffc-cb3c1ec162a3 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:30.576818 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95826d-3bda-470a-8ffc-cb3c1ec162a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:30.576997 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-983d29fd-c33a-4c00-b50c-2193df78eb06 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:31:30.576997 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:30.576997 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:30.576997 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:30.576997 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:30.576997 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:30.576997 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:30.576997 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:30.576997 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:30.576997 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:30.576997 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:30.576997 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:30.576997 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:31:30.576997 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:31:30.576997 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:31:30.576997 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:31:30.578197 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:31:30.578197 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:31:30.578197 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:31:30.578197 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:31:30.578197 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:30.578197 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:30.578197 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:30.578197 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:30.578197 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:30.578197 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:30.578197 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:30.578197 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:30.578197 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:30.578197 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:30.578197 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:30.578197 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:30.579051 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:30.579051 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:30.579051 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:30.579051 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:30.579051 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:31:30.579051 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:31:30.579051 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:31:30.579051 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:31:30.579051 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:31:30.579051 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:31:30.579051 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:31:30.579051 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:30.579652 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 245/489] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Jun 15 14:31:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:30.584211 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983d29fd-c33a-4c00-b50c-2193df78eb06 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:30.585402 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95826d-3bda-470a-8ffc-cb3c1ec162a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:30.600477 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e95826d-3bda-470a-8ffc-cb3c1ec162a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:30.603376 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e95826d-3bda-470a-8ffc-cb3c1ec162a3 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:30.629455 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-9e95826d-3bda-470a-8ffc-cb3c1ec162a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:30.629455 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:30.629455 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:30.629455 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:30.629455 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:30.629455 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:30.629455 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:30.629455 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:30.629455 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:30.629455 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:30.629455 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:30.629455 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:30.629455 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:31:30.629455 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:31:30.629455 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:31:30.630355 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:31:30.630355 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:30.630355 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:30.630355 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:30.630355 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:30.630355 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:30.630355 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:30.630355 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:30.630355 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:30.630355 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:30.630355 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:30.630355 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:30.630355 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:30.630355 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:30.630355 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:30.630355 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:30.630355 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:30.631196 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:31:30.631196 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:31:30.631196 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:30.631196 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:30.631196 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 245/490] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:31:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:30.632503 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e95826d-3bda-470a-8ffc-cb3c1ec162a3 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:30.720313 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983d29fd-c33a-4c00-b50c-2193df78eb06 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:30.758888 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e95826d-3bda-470a-8ffc-cb3c1ec162a3 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:30.762885 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cbfa14-9572-48c5-93c6-30a7a41f0973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:30.763284 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cbfa14-9572-48c5-93c6-30a7a41f0973 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:30.763633 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cbfa14-9572-48c5-93c6-30a7a41f0973 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:30.764004 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cbfa14-9572-48c5-93c6-30a7a41f0973 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:30.775605 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cbfa14-9572-48c5-93c6-30a7a41f0973 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:31:30.783413 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-d3cbfa14-9572-48c5-93c6-30a7a41f0973 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:31:30.783413 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:30.783413 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:30.783413 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:30.783413 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:30.783413 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:30.783413 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:30.783413 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:30.783413 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:30.783413 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:30.783413 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:30.783413 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:30.783413 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:31:30.783413 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:31:30.783413 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:31:30.783413 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:31:30.784333 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:31:30.784333 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:31:30.784333 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:31:30.784333 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:31:30.784333 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:30.784333 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:30.784333 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:30.784333 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:30.784333 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:30.784333 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:30.784333 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:30.784333 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:30.784333 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:30.784333 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:30.784333 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:30.784333 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:30.785147 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:30.785147 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:30.785147 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:30.785147 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:30.785147 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:31:30.785147 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:31:30.785147 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:31:30.785147 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:31:30.785147 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:31:30.785147 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:31:30.785147 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:31:30.785147 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:30.789508 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 246/491] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Jun 15 14:31:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:30.790420 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cbfa14-9572-48c5-93c6-30a7a41f0973 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:30.803428 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1c2d21-08df-45d6-8f22-b31b6bd2f816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:30.804477 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c2d21-08df-45d6-8f22-b31b6bd2f816 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:30.804938 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c2d21-08df-45d6-8f22-b31b6bd2f816 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:30.805388 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c2d21-08df-45d6-8f22-b31b6bd2f816 None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:30.806441 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1c2d21-08df-45d6-8f22-b31b6bd2f816 None admin] RBAC: Adding query filter params (name=service) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:30.806962 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1c2d21-08df-45d6-8f22-b31b6bd2f816 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:30.808777 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1c2d21-08df-45d6-8f22-b31b6bd2f816 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:30.818343 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 246/492] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Jun 15 14:31:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:30.819397 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1c2d21-08df-45d6-8f22-b31b6bd2f816 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:30.834176 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:30.855785 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:30.907031 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cbfa14-9572-48c5-93c6-30a7a41f0973 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:30.942858 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10532fb-49d2-4bdd-9b96-0eb05531df77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:30.943511 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1c2d21-08df-45d6-8f22-b31b6bd2f816 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:30.944019 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10532fb-49d2-4bdd-9b96-0eb05531df77 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:30.944505 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10532fb-49d2-4bdd-9b96-0eb05531df77 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:30.944980 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10532fb-49d2-4bdd-9b96-0eb05531df77 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:30.945877 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10532fb-49d2-4bdd-9b96-0eb05531df77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:30.946359 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10532fb-49d2-4bdd-9b96-0eb05531df77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:30.948013 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10532fb-49d2-4bdd-9b96-0eb05531df77 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:30.961912 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 247/493] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:31:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:30.963033 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10532fb-49d2-4bdd-9b96-0eb05531df77 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:30.987014 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ab489a-7725-4389-9275-4bbe7fedef9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:30.987712 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab489a-7725-4389-9275-4bbe7fedef9d None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:30.988616 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab489a-7725-4389-9275-4bbe7fedef9d None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:30.988946 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab489a-7725-4389-9275-4bbe7fedef9d None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:30.989733 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ab489a-7725-4389-9275-4bbe7fedef9d None admin] RBAC: Adding query filter params (name=service) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:30.990052 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ab489a-7725-4389-9275-4bbe7fedef9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:30.991478 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ab489a-7725-4389-9275-4bbe7fedef9d None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:31.006134 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 247/494] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Jun 15 14:31:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:31.006134 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3306ee-236c-4213-9fd9-fc650435614a None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:31.006134 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3306ee-236c-4213-9fd9-fc650435614a None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:31.006134 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3306ee-236c-4213-9fd9-fc650435614a None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:31.006134 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 248/495] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:31:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:31.006134 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3306ee-236c-4213-9fd9-fc650435614a None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:31.080307 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10532fb-49d2-4bdd-9b96-0eb05531df77 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:31.081520 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:31.096453 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:31.114393 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3306ee-236c-4213-9fd9-fc650435614a None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:31.115177 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639dae43-c59a-4373-adda-4aaf2b5e9396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:31.115969 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639dae43-c59a-4373-adda-4aaf2b5e9396 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:31.116284 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639dae43-c59a-4373-adda-4aaf2b5e9396 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:31.116573 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639dae43-c59a-4373-adda-4aaf2b5e9396 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:31.127867 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639dae43-c59a-4373-adda-4aaf2b5e9396 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:31.129354 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639dae43-c59a-4373-adda-4aaf2b5e9396 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:31.147148 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-639dae43-c59a-4373-adda-4aaf2b5e9396 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:31.147148 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:31.147148 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:31.147148 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:31.147148 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:31.147148 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:31.147148 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:31.147148 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:31.147148 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:31.147148 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:31.147148 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:31.147148 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:31.147148 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:31:31.147148 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:31:31.147148 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:31:31.148010 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:31:31.148010 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:31.148010 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:31.148010 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:31.148010 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:31.148010 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:31.148010 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:31.148010 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:31.148010 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:31.148010 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:31.148010 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:31.148010 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:31.148010 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:31.148010 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:31.148010 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:31.148010 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:31.148010 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:31.148881 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:31:31.148881 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:31:31.148881 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:31.148881 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:31.149098 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769d625d-e60d-415f-bbb3-932a09adcfc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:31.149184 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 248/496] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:31:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:31.149502 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769d625d-e60d-415f-bbb3-932a09adcfc4 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:31.149502 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769d625d-e60d-415f-bbb3-932a09adcfc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:31.149634 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639dae43-c59a-4373-adda-4aaf2b5e9396 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:31.149714 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769d625d-e60d-415f-bbb3-932a09adcfc4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:31.160286 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769d625d-e60d-415f-bbb3-932a09adcfc4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:31.161734 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769d625d-e60d-415f-bbb3-932a09adcfc4 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:31.176694 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-769d625d-e60d-415f-bbb3-932a09adcfc4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:31:31.176694 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:31.176694 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:31.176694 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:31.176694 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:31.176694 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:31.176694 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:31.176694 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:31.176694 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:31.176694 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:31.176694 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:31.176694 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:31.176694 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:31:31.176694 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:31:31.176694 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:31:31.176694 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:31:31.177598 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:31.177598 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:31.177598 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:31:31.177598 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:31:31.177598 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:31:31.177598 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:31:31.177598 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:31.177598 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:31.177598 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:31.177598 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:31.177598 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:31.177598 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:31.177598 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:31.177598 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:31.177598 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:31.177598 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:31.177598 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:31.177598 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:31.178492 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:31.178492 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:31.178492 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:31:31.178492 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:31:31.178492 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:31:31.178492 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:31:31.178492 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:31:31.178492 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:31:31.178492 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:31:31.178492 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:31.178492 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 249/497] 10.222.0.42 () {56 vars in 1093 bytes} [Wed Jun 15 14:31:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:31.179060 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420974c1-ba79-4147-910d-859181a5e579 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:31.179268 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420974c1-ba79-4147-910d-859181a5e579 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:31.179534 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420974c1-ba79-4147-910d-859181a5e579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:31.195839 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-420974c1-ba79-4147-910d-859181a5e579 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:31.197085 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-420974c1-ba79-4147-910d-859181a5e579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:31.254794 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639dae43-c59a-4373-adda-4aaf2b5e9396 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:31.264746 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 250/498] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:31:31.266367 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdc7ba7-5896-403d-a649-a473ec63af97 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:31.266658 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdc7ba7-5896-403d-a649-a473ec63af97 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:31.266935 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdc7ba7-5896-403d-a649-a473ec63af97 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:31.268224 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 251/499] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:31:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:31.268909 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdc7ba7-5896-403d-a649-a473ec63af97 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:31.301854 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c177874-3720-4d86-a358-dd2c0fcffc91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:31.302641 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c177874-3720-4d86-a358-dd2c0fcffc91 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:31.302928 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c177874-3720-4d86-a358-dd2c0fcffc91 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:31.303207 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c177874-3720-4d86-a358-dd2c0fcffc91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:31.313879 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c177874-3720-4d86-a358-dd2c0fcffc91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:31.315302 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c177874-3720-4d86-a358-dd2c0fcffc91 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:31.328845 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:31.332629 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-2c177874-3720-4d86-a358-dd2c0fcffc91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:31.332629 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:31.332629 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:31.332629 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:31.332629 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:31.332629 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:31.332629 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:31.332629 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:31.332629 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:31.332629 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:31.332629 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:31.332629 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:31.332629 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:31:31.332629 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:31:31.332629 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:31:31.333481 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:31:31.333481 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:31.333481 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:31.333481 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:31.333481 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:31.333481 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:31.333481 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:31.333481 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:31.333481 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:31.333481 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:31.333481 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:31.333481 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:31.333481 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:31.333481 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:31.333481 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:31.333481 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:31.333481 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:31.334308 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:31:31.334308 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:31:31.334308 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:31.334308 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:31.334308 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 249/500] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:31:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:31.334603 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c177874-3720-4d86-a358-dd2c0fcffc91 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:31.355826 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:31.374593 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdc7ba7-5896-403d-a649-a473ec63af97 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:31.409554 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b55922-c60a-4467-89c7-d72393812117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:31.410208 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b55922-c60a-4467-89c7-d72393812117 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:31.410489 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b55922-c60a-4467-89c7-d72393812117 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:31.410760 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b55922-c60a-4467-89c7-d72393812117 None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:31.411469 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b55922-c60a-4467-89c7-d72393812117 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:31.411771 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b55922-c60a-4467-89c7-d72393812117 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:31.413165 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b55922-c60a-4467-89c7-d72393812117 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:31.424650 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 252/501] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:31:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:31.425978 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2242f5-1c5f-4060-aa4e-2f007b39ea3d None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:31.426266 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2242f5-1c5f-4060-aa4e-2f007b39ea3d None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:31.426631 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2242f5-1c5f-4060-aa4e-2f007b39ea3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:31.433487 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c177874-3720-4d86-a358-dd2c0fcffc91 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:31.442608 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-5b2242f5-1c5f-4060-aa4e-2f007b39ea3d None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:31.443778 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-5b2242f5-1c5f-4060-aa4e-2f007b39ea3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:31.467931 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261ed826-e2f0-42fc-92ce-ec2e2ede8530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:31.468634 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ed826-e2f0-42fc-92ce-ec2e2ede8530 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:31.468917 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ed826-e2f0-42fc-92ce-ec2e2ede8530 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:31.469197 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ed826-e2f0-42fc-92ce-ec2e2ede8530 None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:31.470034 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261ed826-e2f0-42fc-92ce-ec2e2ede8530 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:31.470345 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261ed826-e2f0-42fc-92ce-ec2e2ede8530 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:31.471779 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261ed826-e2f0-42fc-92ce-ec2e2ede8530 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:31.514340 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 250/503] 10.222.0.42 () {56 vars in 1133 bytes} [Wed Jun 15 14:31:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:31.514519 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 253/503] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:31:31.514519 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2242f5-1c5f-4060-aa4e-2f007b39ea3d None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:31.514683 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261ed826-e2f0-42fc-92ce-ec2e2ede8530 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:31.566806 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:31.596873 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:31.615142 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2242f5-1c5f-4060-aa4e-2f007b39ea3d None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:31.618838 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261ed826-e2f0-42fc-92ce-ec2e2ede8530 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:31.649858 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d795425b-6137-428c-aac1-0361a6a4c6f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:31.650936 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d795425b-6137-428c-aac1-0361a6a4c6f6 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:31.651217 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d795425b-6137-428c-aac1-0361a6a4c6f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:31.651498 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d795425b-6137-428c-aac1-0361a6a4c6f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:31.655858 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe53f3b-30e0-4ca1-a6b9-a93da3eed957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:31.656558 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe53f3b-30e0-4ca1-a6b9-a93da3eed957 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:31.656839 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe53f3b-30e0-4ca1-a6b9-a93da3eed957 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:31.657114 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe53f3b-30e0-4ca1-a6b9-a93da3eed957 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:31.657833 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe53f3b-30e0-4ca1-a6b9-a93da3eed957 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:31.658138 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe53f3b-30e0-4ca1-a6b9-a93da3eed957 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:31.659516 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe53f3b-30e0-4ca1-a6b9-a93da3eed957 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:31.662151 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d795425b-6137-428c-aac1-0361a6a4c6f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:31.663568 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d795425b-6137-428c-aac1-0361a6a4c6f6 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:31.671063 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 251/504] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:31:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:31.671849 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe53f3b-30e0-4ca1-a6b9-a93da3eed957 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:31.684642 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-d795425b-6137-428c-aac1-0361a6a4c6f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:31.684642 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:31.684642 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:31.684642 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:31.684642 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:31.684642 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:31.684642 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:31.684642 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:31.684642 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:31.684642 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:31.684642 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:31.684642 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:31.684642 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:31:31.684642 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:31:31.684642 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:31:31.685627 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:31:31.685627 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:31.685627 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:31.685627 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:31.685627 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:31.685627 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:31.685627 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:31.685627 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:31.685627 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:31.685627 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:31.685627 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:31.685627 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:31.685627 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:31.685627 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:31.685627 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:31.685627 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:31.685627 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:31.687140 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:31:31.687140 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:31:31.687140 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:31.687140 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:31.687140 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 254/505] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:31:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:31.687140 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d795425b-6137-428c-aac1-0361a6a4c6f6 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:31.769068 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe53f3b-30e0-4ca1-a6b9-a93da3eed957 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:31.784921 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d795425b-6137-428c-aac1-0361a6a4c6f6 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:31.786708 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:31.802727 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b006dd-0533-4c53-9d2e-51517a958633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:31.803617 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b006dd-0533-4c53-9d2e-51517a958633 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:31.803972 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b006dd-0533-4c53-9d2e-51517a958633 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:31.804333 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b006dd-0533-4c53-9d2e-51517a958633 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:31.811888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b006dd-0533-4c53-9d2e-51517a958633 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:31.813348 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b006dd-0533-4c53-9d2e-51517a958633 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:31.815165 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:31.820515 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623f652e-92eb-4845-b14b-bc46d881e696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:31.821152 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623f652e-92eb-4845-b14b-bc46d881e696 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:31.825803 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623f652e-92eb-4845-b14b-bc46d881e696 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:31.825803 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623f652e-92eb-4845-b14b-bc46d881e696 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:31.829247 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-09b006dd-0533-4c53-9d2e-51517a958633 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 14:31:31.829247 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:31.829247 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:31.829247 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:31.829247 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:31.829247 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:31.829247 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:31.829247 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:31.829247 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:31.829247 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:31.829247 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:31.829247 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:31.829247 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:31:31.829247 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:31:31.829247 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:31:31.829247 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:31:31.830244 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:31.830244 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:31.830244 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:31:31.830244 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:31:31.830244 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:31:31.830244 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:31:31.830244 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:31.830244 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:31.830244 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:31.830244 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:31.830244 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:31.830244 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:31.830244 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:31.830244 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:31.830244 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:31.830244 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:31.830244 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:31.830244 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:31.831183 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:31.831183 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:31.831183 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:31:31.831183 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:31:31.831183 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:31:31.831183 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:31:31.831183 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:31:31.831183 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:31:31.831183 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 14:31:31.831183 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:31.831183 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 252/506] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jun 15 14:31:31 2022] GET /identity/v3/users/cinder => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:31.831746 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b006dd-0533-4c53-9d2e-51517a958633 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:31.890595 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623f652e-92eb-4845-b14b-bc46d881e696 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:31.892053 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623f652e-92eb-4845-b14b-bc46d881e696 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:31.934353 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b006dd-0533-4c53-9d2e-51517a958633 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:31.965019 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd10a575-a344-4d8d-87b7-cb3231a75a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:31.968842 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd10a575-a344-4d8d-87b7-cb3231a75a6d None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:31.968842 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd10a575-a344-4d8d-87b7-cb3231a75a6d None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:31.968842 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd10a575-a344-4d8d-87b7-cb3231a75a6d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:31.977105 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd10a575-a344-4d8d-87b7-cb3231a75a6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:31:31.985155 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-bd10a575-a344-4d8d-87b7-cb3231a75a6d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:31:31.985155 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:31.985155 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:31.985155 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:31.985155 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:31.985155 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:31.985155 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:31.985155 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:31.985155 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:31.985155 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:31.985155 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:31.985155 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:31.985155 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:31:31.985155 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:31:31.985155 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:31:31.985155 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:31:31.986176 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:31:31.986176 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:31:31.986176 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:31:31.986176 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:31:31.986176 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:31.986176 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:31.986176 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:31.986176 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:31.986176 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:31.986176 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:31.986176 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:31.986176 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:31.986176 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:31.986176 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:31.986176 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:31.986176 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:31.987137 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:31.987137 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:31.987137 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:31.987137 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:31.987137 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:31:31.987137 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:31:31.987137 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:31:31.987137 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:31:31.987137 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:31:31.987137 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:31:31.987137 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:31:31.987137 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:31.987137 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 253/507] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Jun 15 14:31:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:31.987137 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd10a575-a344-4d8d-87b7-cb3231a75a6d None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:31.991187 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 255/508] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jun 15 14:31:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:31:31.992014 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623f652e-92eb-4845-b14b-bc46d881e696 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:31.994588 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:32.048542 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:32.090802 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd10a575-a344-4d8d-87b7-cb3231a75a6d None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:32.093014 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623f652e-92eb-4845-b14b-bc46d881e696 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:32.126052 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703a20ee-525c-4793-8dcc-b67dd34f2142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:32.126611 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4999f028-c9a6-4ffc-ba7f-5a461098ea09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:32.126720 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703a20ee-525c-4793-8dcc-b67dd34f2142 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:32.126973 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703a20ee-525c-4793-8dcc-b67dd34f2142 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:32.127249 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703a20ee-525c-4793-8dcc-b67dd34f2142 None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:32.127395 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4999f028-c9a6-4ffc-ba7f-5a461098ea09 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:32.127681 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4999f028-c9a6-4ffc-ba7f-5a461098ea09 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:32.127959 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703a20ee-525c-4793-8dcc-b67dd34f2142 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:32.128045 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4999f028-c9a6-4ffc-ba7f-5a461098ea09 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:32.128298 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703a20ee-525c-4793-8dcc-b67dd34f2142 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:32.129670 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703a20ee-525c-4793-8dcc-b67dd34f2142 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:32.135492 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4999f028-c9a6-4ffc-ba7f-5a461098ea09 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:32.136949 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4999f028-c9a6-4ffc-ba7f-5a461098ea09 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:32.141929 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 256/509] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:31:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:32.142767 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703a20ee-525c-4793-8dcc-b67dd34f2142 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:32.151233 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-4999f028-c9a6-4ffc-ba7f-5a461098ea09 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 15 14:31:32.151233 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:32.151233 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:32.151233 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:32.151233 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:32.151233 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:32.151233 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:32.151233 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:32.151233 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:32.151233 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:32.151233 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:32.151233 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:32.151233 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:31:32.151233 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:31:32.151233 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:31:32.151233 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:31:32.152318 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:32.152318 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:32.152318 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:31:32.152318 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:31:32.152318 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:31:32.152318 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:31:32.152318 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:32.152318 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:32.152318 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:32.152318 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:32.152318 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:32.152318 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:32.152318 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:32.152318 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:32.152318 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:32.152318 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:32.152318 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:32.152318 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:32.153439 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:32.153439 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:32.153439 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:31:32.153439 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:31:32.153439 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:31:32.153439 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:31:32.153439 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:31:32.153439 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:31:32.153439 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 15 14:31:32.153439 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:32.153439 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 254/510] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jun 15 14:31:31 2022] GET /identity/v3/users/glance => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:32.153439 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4999f028-c9a6-4ffc-ba7f-5a461098ea09 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:32.208348 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:32.245835 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703a20ee-525c-4793-8dcc-b67dd34f2142 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:32.255366 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4999f028-c9a6-4ffc-ba7f-5a461098ea09 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:32.271972 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:32.279744 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe483e7-811f-4bf5-84e9-211953c1d8f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:32.280512 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe483e7-811f-4bf5-84e9-211953c1d8f2 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:32.280854 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe483e7-811f-4bf5-84e9-211953c1d8f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:32.281186 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe483e7-811f-4bf5-84e9-211953c1d8f2 None admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:32.282127 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe483e7-811f-4bf5-84e9-211953c1d8f2 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:32.282508 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe483e7-811f-4bf5-84e9-211953c1d8f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:32.283925 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe483e7-811f-4bf5-84e9-211953c1d8f2 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:32.288959 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19083115-0114-487b-ba96-2d60aa9b4d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:32.289610 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19083115-0114-487b-ba96-2d60aa9b4d6b None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:32.289894 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19083115-0114-487b-ba96-2d60aa9b4d6b None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:32.290170 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19083115-0114-487b-ba96-2d60aa9b4d6b None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:32.290914 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19083115-0114-487b-ba96-2d60aa9b4d6b None admin] RBAC: Adding query filter params (name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:32.291226 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19083115-0114-487b-ba96-2d60aa9b4d6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:32.292614 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19083115-0114-487b-ba96-2d60aa9b4d6b None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:32.301882 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 255/511] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Jun 15 14:31:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:32.302947 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19083115-0114-487b-ba96-2d60aa9b4d6b None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:32.320497 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 257/512] 10.222.0.42 () {56 vars in 1137 bytes} [Wed Jun 15 14:31:32 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:32.321304 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe483e7-811f-4bf5-84e9-211953c1d8f2 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:32.404690 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19083115-0114-487b-ba96-2d60aa9b4d6b None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:32.420295 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe483e7-811f-4bf5-84e9-211953c1d8f2 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:32.437693 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e259bf66-9808-4488-9c6d-be9cb819d892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:32.438475 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e259bf66-9808-4488-9c6d-be9cb819d892 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:32.438761 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e259bf66-9808-4488-9c6d-be9cb819d892 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:32.439038 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e259bf66-9808-4488-9c6d-be9cb819d892 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:32.439741 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:32.448929 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e259bf66-9808-4488-9c6d-be9cb819d892 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:32.450446 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e259bf66-9808-4488-9c6d-be9cb819d892 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:32.453746 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa40b507-08a1-4d00-87aa-6bcae0764010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:32.454490 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa40b507-08a1-4d00-87aa-6bcae0764010 None admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:32.454768 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa40b507-08a1-4d00-87aa-6bcae0764010 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:32.455040 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa40b507-08a1-4d00-87aa-6bcae0764010 None admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:32.455728 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa40b507-08a1-4d00-87aa-6bcae0764010 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:32.457192 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa40b507-08a1-4d00-87aa-6bcae0764010 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:32.468757 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-e259bf66-9808-4488-9c6d-be9cb819d892 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:32.468757 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:32.468757 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:32.468757 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:32.468757 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:32.468757 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:32.468757 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:32.468757 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:32.468757 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:32.468757 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:32.468757 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:32.468757 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:32.468757 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:31:32.468757 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:31:32.468757 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:31:32.469756 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:31:32.469756 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:32.469756 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:32.469756 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:32.469756 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:32.469756 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:32.469756 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:32.469756 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:32.469756 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:32.469756 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:32.469756 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:32.469756 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:32.469756 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:32.469756 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:32.469756 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:32.469756 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:32.469756 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:32.475542 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:31:32.475542 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:31:32.475542 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:32.475542 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:32.475542 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 256/513] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:31:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:32.475542 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e259bf66-9808-4488-9c6d-be9cb819d892 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:32.501686 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:32.536287 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 258/514] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Jun 15 14:31:32 2022] POST /identity/v3/users => generated 247 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:31:32.537098 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa40b507-08a1-4d00-87aa-6bcae0764010 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:32.583459 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e259bf66-9808-4488-9c6d-be9cb819d892 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:32.622244 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e25131-8027-4004-b398-c3c0b6ae484a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:32.623248 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e25131-8027-4004-b398-c3c0b6ae484a None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:32.623515 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e25131-8027-4004-b398-c3c0b6ae484a None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:32.623802 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e25131-8027-4004-b398-c3c0b6ae484a None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:32.624714 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e25131-8027-4004-b398-c3c0b6ae484a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:32.625033 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e25131-8027-4004-b398-c3c0b6ae484a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:32.626519 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e25131-8027-4004-b398-c3c0b6ae484a None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:32.646337 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 257/515] 10.222.0.42 () {56 vars in 1137 bytes} [Wed Jun 15 14:31:32 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:32.647218 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e25131-8027-4004-b398-c3c0b6ae484a None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:32.660145 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa40b507-08a1-4d00-87aa-6bcae0764010 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:32.697692 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa59ed6-3584-4deb-91fb-15fc9609a42a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:32.698923 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa59ed6-3584-4deb-91fb-15fc9609a42a None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:32.698923 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa59ed6-3584-4deb-91fb-15fc9609a42a None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:32.699207 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa59ed6-3584-4deb-91fb-15fc9609a42a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:32.712294 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa59ed6-3584-4deb-91fb-15fc9609a42a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:32.713944 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa59ed6-3584-4deb-91fb-15fc9609a42a None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:32.734737 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-5fa59ed6-3584-4deb-91fb-15fc9609a42a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:32.734737 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:32.734737 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:32.734737 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:32.734737 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:32.734737 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:32.734737 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:32.734737 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:32.734737 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:32.734737 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:32.734737 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:32.734737 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:32.734737 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:31:32.734737 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:31:32.734737 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:31:32.735675 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:31:32.735675 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:32.735675 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:32.735675 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:32.735675 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:32.735675 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:32.735675 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:32.735675 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:32.735675 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:32.735675 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:32.735675 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:32.735675 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:32.735675 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:32.735675 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:32.735675 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:32.735675 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:32.735675 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:32.736796 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:31:32.736796 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:31:32.736796 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:32.736796 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:32.736796 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 259/516] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:31:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:32.738624 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa59ed6-3584-4deb-91fb-15fc9609a42a None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:32.755512 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:32.767268 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e25131-8027-4004-b398-c3c0b6ae484a None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:32.775744 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:32.808403 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e6b8d3-441e-4221-8217-40f6407d7b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:32.809054 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e6b8d3-441e-4221-8217-40f6407d7b35 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:32.809682 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e6b8d3-441e-4221-8217-40f6407d7b35 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:32.810270 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e6b8d3-441e-4221-8217-40f6407d7b35 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:32.851944 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa59ed6-3584-4deb-91fb-15fc9609a42a None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:32.887190 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e6b8d3-441e-4221-8217-40f6407d7b35 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:32.888834 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e6b8d3-441e-4221-8217-40f6407d7b35 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:32.888975 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a709551-9f6b-42ca-84c7-10ac1c321b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:32.889494 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a709551-9f6b-42ca-84c7-10ac1c321b18 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:32.889777 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a709551-9f6b-42ca-84c7-10ac1c321b18 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:32.890054 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a709551-9f6b-42ca-84c7-10ac1c321b18 None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:32.890776 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a709551-9f6b-42ca-84c7-10ac1c321b18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:32.891180 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a709551-9f6b-42ca-84c7-10ac1c321b18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:32.892853 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a709551-9f6b-42ca-84c7-10ac1c321b18 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:32.905329 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 260/517] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:31:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:32.906152 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a709551-9f6b-42ca-84c7-10ac1c321b18 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:32.993337 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:32.998511 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:33.005447 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 258/518] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jun 15 14:31:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:31:33.006460 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e6b8d3-441e-4221-8217-40f6407d7b35 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:33.017892 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a709551-9f6b-42ca-84c7-10ac1c321b18 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:33.054668 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f8aebc-7e8d-43db-97d9-a723de3ff7b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:33.055473 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f8aebc-7e8d-43db-97d9-a723de3ff7b1 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:33.055752 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f8aebc-7e8d-43db-97d9-a723de3ff7b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:33.056419 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f8aebc-7e8d-43db-97d9-a723de3ff7b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:33.121385 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e6b8d3-441e-4221-8217-40f6407d7b35 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:33.130068 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f8aebc-7e8d-43db-97d9-a723de3ff7b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:33.131939 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f8aebc-7e8d-43db-97d9-a723de3ff7b1 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:33.156561 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6192f65b-f123-43fe-beb5-84de270fecb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:33.157212 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6192f65b-f123-43fe-beb5-84de270fecb6 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:33.157497 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6192f65b-f123-43fe-beb5-84de270fecb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:33.157766 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6192f65b-f123-43fe-beb5-84de270fecb6 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:33.158484 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6192f65b-f123-43fe-beb5-84de270fecb6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:33.158785 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6192f65b-f123-43fe-beb5-84de270fecb6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:33.160147 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6192f65b-f123-43fe-beb5-84de270fecb6 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:33.173657 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 259/519] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:31:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:33.178071 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6192f65b-f123-43fe-beb5-84de270fecb6 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:33.226575 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:33.230614 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:33.248262 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 261/520] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jun 15 14:31:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:31:33.248262 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f8aebc-7e8d-43db-97d9-a723de3ff7b1 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:33.283450 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6192f65b-f123-43fe-beb5-84de270fecb6 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:33.319057 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d514e0a-1f0e-47a7-a2d9-74e1bea7ff5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:33.319847 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d514e0a-1f0e-47a7-a2d9-74e1bea7ff5b None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:33.320143 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d514e0a-1f0e-47a7-a2d9-74e1bea7ff5b None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:33.320422 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d514e0a-1f0e-47a7-a2d9-74e1bea7ff5b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:33.328416 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d514e0a-1f0e-47a7-a2d9-74e1bea7ff5b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:33.329853 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d514e0a-1f0e-47a7-a2d9-74e1bea7ff5b None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:33.344562 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-6d514e0a-1f0e-47a7-a2d9-74e1bea7ff5b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:31:33.344562 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:33.344562 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:33.344562 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:33.344562 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:33.344562 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:33.344562 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:33.344562 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:33.344562 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:33.344562 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:33.344562 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:33.344562 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:33.344562 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:31:33.344562 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:31:33.344562 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:31:33.345640 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:31:33.345640 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:33.345640 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:33.345640 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:33.345640 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:33.345640 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:33.345640 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:33.345640 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:33.345640 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:33.345640 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:33.345640 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:33.345640 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:33.345640 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:33.345640 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:33.345640 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:33.345640 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:33.345640 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:33.346658 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:31:33.346658 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:31:33.346658 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:31:33.346658 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:31:33.346658 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:31:33.346658 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:31:33.346658 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:31:33.346658 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:33.346658 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 260/521] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jun 15 14:31:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:33.350983 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d514e0a-1f0e-47a7-a2d9-74e1bea7ff5b None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:33.356389 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f8aebc-7e8d-43db-97d9-a723de3ff7b1 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:33.394297 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4033b6bf-8482-41e5-87a7-09140bcfedde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:33.395167 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4033b6bf-8482-41e5-87a7-09140bcfedde None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:33.395515 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4033b6bf-8482-41e5-87a7-09140bcfedde None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:33.395789 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4033b6bf-8482-41e5-87a7-09140bcfedde None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:33.407299 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4033b6bf-8482-41e5-87a7-09140bcfedde None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:33.408930 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4033b6bf-8482-41e5-87a7-09140bcfedde None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:33.426743 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-4033b6bf-8482-41e5-87a7-09140bcfedde None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:33.426743 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:33.426743 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:33.426743 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:33.426743 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:33.426743 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:33.426743 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:33.426743 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:33.426743 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:33.426743 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:33.426743 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:33.426743 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:33.426743 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:31:33.426743 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:31:33.426743 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:31:33.427972 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:31:33.427972 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:33.427972 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:33.427972 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:33.427972 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:33.427972 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:33.427972 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:33.427972 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:33.427972 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:33.427972 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:33.427972 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:33.427972 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:33.427972 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:33.427972 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:33.427972 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:33.427972 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:33.427972 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:33.429071 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:31:33.429071 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:31:33.429071 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:33.429071 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:33.432658 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 262/522] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:31:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:33.433648 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4033b6bf-8482-41e5-87a7-09140bcfedde None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:33.457399 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d514e0a-1f0e-47a7-a2d9-74e1bea7ff5b None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:33.469379 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:33.479329 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:33.492557 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475cf0c1-89b4-473b-8dab-90d2d22425c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:33.493376 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475cf0c1-89b4-473b-8dab-90d2d22425c6 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:33.493668 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475cf0c1-89b4-473b-8dab-90d2d22425c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:33.493947 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475cf0c1-89b4-473b-8dab-90d2d22425c6 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:33.502271 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475cf0c1-89b4-473b-8dab-90d2d22425c6 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:33.503737 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475cf0c1-89b4-473b-8dab-90d2d22425c6 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:33.520850 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-475cf0c1-89b4-473b-8dab-90d2d22425c6 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 15 14:31:33.520850 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:33.520850 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:33.520850 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:33.520850 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:33.520850 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:33.520850 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:33.520850 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:33.520850 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:33.520850 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:33.520850 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:33.520850 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:33.520850 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:31:33.520850 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:31:33.520850 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:31:33.520850 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:31:33.521763 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:33.521763 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:33.521763 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:31:33.521763 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:31:33.521763 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:31:33.521763 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:31:33.521763 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:33.521763 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:33.521763 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:33.521763 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:33.521763 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:33.521763 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:33.521763 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:33.521763 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:33.521763 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:33.521763 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:33.521763 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:33.521763 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:33.522636 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:33.522636 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:33.522636 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:31:33.522636 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:31:33.522636 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:31:33.522636 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:31:33.522636 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:31:33.522636 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:31:33.522636 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 15 14:31:33.522636 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:33.522636 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 261/523] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Jun 15 14:31:33 2022] GET /identity/v3/users/neutron => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:33.523309 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475cf0c1-89b4-473b-8dab-90d2d22425c6 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:33.553635 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4033b6bf-8482-41e5-87a7-09140bcfedde None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:33.588290 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38897bcf-b948-41f6-903b-33c5f2b416d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:33.589164 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38897bcf-b948-41f6-903b-33c5f2b416d1 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:33.589513 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38897bcf-b948-41f6-903b-33c5f2b416d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:33.589854 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38897bcf-b948-41f6-903b-33c5f2b416d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:33.601310 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38897bcf-b948-41f6-903b-33c5f2b416d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:33.602769 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38897bcf-b948-41f6-903b-33c5f2b416d1 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:33.620927 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-38897bcf-b948-41f6-903b-33c5f2b416d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:33.620927 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:33.620927 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:33.620927 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:33.620927 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:33.620927 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:33.620927 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:33.620927 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:33.620927 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:33.620927 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:33.620927 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:33.620927 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:33.620927 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:31:33.620927 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:31:33.620927 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:31:33.622601 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:31:33.622601 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:33.622601 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:33.622601 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:33.622601 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:33.622601 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:33.622601 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:33.622601 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:33.622601 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:33.622601 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:33.622601 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:33.622601 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:33.622601 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:33.622601 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:33.622601 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:33.622601 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:33.622601 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:33.623581 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:31:33.623581 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:31:33.623581 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:33.623581 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:33.623581 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 263/524] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:31:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:33.623581 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38897bcf-b948-41f6-903b-33c5f2b416d1 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:33.640276 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475cf0c1-89b4-473b-8dab-90d2d22425c6 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:33.677292 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6ef960-92c6-456c-a453-204019a2c38f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:33.677972 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6ef960-92c6-456c-a453-204019a2c38f None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:33.678265 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6ef960-92c6-456c-a453-204019a2c38f None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:33.678544 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6ef960-92c6-456c-a453-204019a2c38f None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:33.679362 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6ef960-92c6-456c-a453-204019a2c38f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:33.679677 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6ef960-92c6-456c-a453-204019a2c38f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:33.681211 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6ef960-92c6-456c-a453-204019a2c38f None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:33.702453 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 262/525] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jun 15 14:31:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:33.703312 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6ef960-92c6-456c-a453-204019a2c38f None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:33.725093 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:33.734784 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38897bcf-b948-41f6-903b-33c5f2b416d1 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:33.739044 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:33.772800 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b628ac-8fc2-45d7-898f-63776db743ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:33.773462 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b628ac-8fc2-45d7-898f-63776db743ff None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:33.773744 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b628ac-8fc2-45d7-898f-63776db743ff None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:33.774019 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b628ac-8fc2-45d7-898f-63776db743ff None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:33.774984 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b628ac-8fc2-45d7-898f-63776db743ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:33.775296 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b628ac-8fc2-45d7-898f-63776db743ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:33.776678 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b628ac-8fc2-45d7-898f-63776db743ff None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:33.792695 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 264/526] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:31:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:33.792695 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b628ac-8fc2-45d7-898f-63776db743ff None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:33.817660 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6ef960-92c6-456c-a453-204019a2c38f None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:33.853620 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1279e376-a1de-4e7b-b2ea-87ac33cc300a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:33.854359 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1279e376-a1de-4e7b-b2ea-87ac33cc300a None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:33.854737 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1279e376-a1de-4e7b-b2ea-87ac33cc300a None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:33.855079 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1279e376-a1de-4e7b-b2ea-87ac33cc300a None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:33.856718 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1279e376-a1de-4e7b-b2ea-87ac33cc300a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:33.856933 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1279e376-a1de-4e7b-b2ea-87ac33cc300a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:33.858422 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1279e376-a1de-4e7b-b2ea-87ac33cc300a None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:33.881394 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 263/527] 10.222.0.42 () {56 vars in 1139 bytes} [Wed Jun 15 14:31:33 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:33.881394 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1279e376-a1de-4e7b-b2ea-87ac33cc300a None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:33.899145 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b628ac-8fc2-45d7-898f-63776db743ff None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:33.937314 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2539c7cc-f3d4-40f3-85ff-39504dd8fd1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:33.948023 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2539c7cc-f3d4-40f3-85ff-39504dd8fd1c None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:33.948023 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2539c7cc-f3d4-40f3-85ff-39504dd8fd1c None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:33.948023 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2539c7cc-f3d4-40f3-85ff-39504dd8fd1c None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:33.948023 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2539c7cc-f3d4-40f3-85ff-39504dd8fd1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:33.948023 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2539c7cc-f3d4-40f3-85ff-39504dd8fd1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:33.948023 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2539c7cc-f3d4-40f3-85ff-39504dd8fd1c None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:33.959079 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 265/528] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:31:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:33.959079 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2539c7cc-f3d4-40f3-85ff-39504dd8fd1c None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:33.970577 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:33.984269 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:33.996485 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1279e376-a1de-4e7b-b2ea-87ac33cc300a None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:34.031870 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4303504a-a08e-49a8-879c-84baad3961ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:34.032471 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4303504a-a08e-49a8-879c-84baad3961ea None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:34.032750 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4303504a-a08e-49a8-879c-84baad3961ea None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:34.033022 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4303504a-a08e-49a8-879c-84baad3961ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:34.033834 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4303504a-a08e-49a8-879c-84baad3961ea None admin] RBAC: Adding query filter params (role.id=305a1ca2e6f740eba7dc696849e933eb, scope.project.id=3094ff8966f149bda784da6a51b1358f, user.id=0927c05c340c48e48b18754888aa9c35) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:34.034142 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4303504a-a08e-49a8-879c-84baad3961ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:34.035551 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4303504a-a08e-49a8-879c-84baad3961ea None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:34.041504 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 264/529] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jun 15 14:31:33 2022] GET /identity/v3/role_assignments?user.id=0927c05c340c48e48b18754888aa9c35&scope.project.id=3094ff8966f149bda784da6a51b1358f&role.id=305a1ca2e6f740eba7dc696849e933eb => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:34.049825 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4303504a-a08e-49a8-879c-84baad3961ea None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:34.063633 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2539c7cc-f3d4-40f3-85ff-39504dd8fd1c None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:34.102385 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6140722-55ee-4210-82d7-80af07e6fb2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:34.103264 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6140722-55ee-4210-82d7-80af07e6fb2a None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:34.103746 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6140722-55ee-4210-82d7-80af07e6fb2a None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:34.103976 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6140722-55ee-4210-82d7-80af07e6fb2a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:34.113338 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6140722-55ee-4210-82d7-80af07e6fb2a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:34.114433 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6140722-55ee-4210-82d7-80af07e6fb2a None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:34.131193 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-d6140722-55ee-4210-82d7-80af07e6fb2a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:31:34.131193 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:34.131193 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:34.131193 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:34.131193 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:34.131193 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:34.131193 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:34.131193 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:34.131193 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:34.131193 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:34.131193 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:34.131193 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:34.131193 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:31:34.131193 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:31:34.131193 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:31:34.132162 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:31:34.132162 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:34.132162 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:34.132162 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:34.132162 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:34.132162 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:34.132162 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:34.132162 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:34.132162 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:34.132162 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:34.132162 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:34.132162 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:34.132162 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:34.132162 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:34.132162 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:34.132162 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:34.132162 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:34.133031 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:31:34.133031 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:31:34.133031 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:31:34.133031 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:31:34.133031 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:31:34.133031 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:31:34.133031 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:31:34.133031 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:34.137638 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 266/530] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jun 15 14:31:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:34.138866 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6140722-55ee-4210-82d7-80af07e6fb2a None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:34.201996 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4303504a-a08e-49a8-879c-84baad3961ea None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:34.223444 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:34.249316 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f66461-1853-4f5d-908e-225a7ad58082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:34.250570 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f66461-1853-4f5d-908e-225a7ad58082 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:34.251220 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f66461-1853-4f5d-908e-225a7ad58082 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:34.251993 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f66461-1853-4f5d-908e-225a7ad58082 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:34.255072 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6140722-55ee-4210-82d7-80af07e6fb2a None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:34.282538 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:34.301948 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b3488-a723-436e-a41f-6fd12d797626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:34.303355 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b3488-a723-436e-a41f-6fd12d797626 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:34.303355 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b3488-a723-436e-a41f-6fd12d797626 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:34.303355 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b3488-a723-436e-a41f-6fd12d797626 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:34.311277 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723b3488-a723-436e-a41f-6fd12d797626 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:34.312769 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723b3488-a723-436e-a41f-6fd12d797626 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:34.327572 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-723b3488-a723-436e-a41f-6fd12d797626 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:31:34.327572 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:34.327572 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:34.327572 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:34.327572 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:34.327572 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:34.327572 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:34.327572 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:34.327572 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:34.327572 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:34.327572 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:34.327572 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:34.327572 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:31:34.327572 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:31:34.327572 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:31:34.328518 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:31:34.328518 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:34.328518 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:34.328518 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:34.328518 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:34.328518 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:34.328518 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:34.328518 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:34.328518 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:34.328518 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:34.328518 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:34.328518 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:34.328518 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:34.328518 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:34.328518 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:34.328518 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:34.328518 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:34.329367 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:31:34.329367 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:31:34.329367 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:31:34.329367 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:31:34.329367 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:31:34.329367 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:31:34.329367 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:31:34.329367 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:34.333561 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 267/531] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jun 15 14:31:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:34.334283 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b3488-a723-436e-a41f-6fd12d797626 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:34.350055 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f66461-1853-4f5d-908e-225a7ad58082 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:34.352635 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f66461-1853-4f5d-908e-225a7ad58082 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:34.443053 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b3488-a723-436e-a41f-6fd12d797626 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:34.463636 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 265/532] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jun 15 14:31:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:31:34.464828 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f66461-1853-4f5d-908e-225a7ad58082 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:34.478766 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:34.485034 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7473784a-72f2-47ef-aedf-5e48f05a0174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:34.485922 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7473784a-72f2-47ef-aedf-5e48f05a0174 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:34.486045 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7473784a-72f2-47ef-aedf-5e48f05a0174 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:34.486230 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7473784a-72f2-47ef-aedf-5e48f05a0174 None admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:34.487018 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7473784a-72f2-47ef-aedf-5e48f05a0174 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:34.487325 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7473784a-72f2-47ef-aedf-5e48f05a0174 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:34.488845 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7473784a-72f2-47ef-aedf-5e48f05a0174 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:34.509386 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 268/533] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jun 15 14:31:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:34.515798 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7473784a-72f2-47ef-aedf-5e48f05a0174 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:34.548572 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:34.583040 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f66461-1853-4f5d-908e-225a7ad58082 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:34.618722 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd221efa-bbde-472e-9566-5019de72afbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:34.619434 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd221efa-bbde-472e-9566-5019de72afbb None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:34.619755 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd221efa-bbde-472e-9566-5019de72afbb None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:34.620037 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd221efa-bbde-472e-9566-5019de72afbb None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:34.620851 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd221efa-bbde-472e-9566-5019de72afbb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:34.621160 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd221efa-bbde-472e-9566-5019de72afbb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:34.623270 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd221efa-bbde-472e-9566-5019de72afbb None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:34.635072 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 266/534] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jun 15 14:31:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:34.641261 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd221efa-bbde-472e-9566-5019de72afbb None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:34.649972 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7473784a-72f2-47ef-aedf-5e48f05a0174 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:34.696015 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed534a0-3a4a-4859-b65f-9f608e77ebdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:34.697366 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed534a0-3a4a-4859-b65f-9f608e77ebdd None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:34.698150 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed534a0-3a4a-4859-b65f-9f608e77ebdd None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:34.698902 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed534a0-3a4a-4859-b65f-9f608e77ebdd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:34.715242 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed534a0-3a4a-4859-b65f-9f608e77ebdd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:34.717169 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed534a0-3a4a-4859-b65f-9f608e77ebdd None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:34.737103 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-eed534a0-3a4a-4859-b65f-9f608e77ebdd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:34.737103 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:34.737103 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:34.737103 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:34.737103 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:34.737103 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:34.737103 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:34.737103 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:34.737103 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:34.737103 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:34.737103 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:34.737103 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:34.737103 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:31:34.737103 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:31:34.737103 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:31:34.738043 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:31:34.738043 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:34.738043 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:34.738043 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:34.738043 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:34.738043 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:34.738043 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:34.738043 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:34.738043 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:34.738043 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:34.738043 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:34.738043 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:34.738043 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:34.738043 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:34.738043 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:34.738043 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:34.738043 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:34.738961 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:31:34.738961 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:31:34.738961 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:34.738961 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:34.738961 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 269/535] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:31:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:34.743766 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed534a0-3a4a-4859-b65f-9f608e77ebdd None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:34.757920 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd221efa-bbde-472e-9566-5019de72afbb None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:34.780387 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:34.795548 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:34.802680 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3a450e-1d3c-4ea3-a432-3e57dbeb27b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:34.803268 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3a450e-1d3c-4ea3-a432-3e57dbeb27b3 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:34.803548 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3a450e-1d3c-4ea3-a432-3e57dbeb27b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:34.803822 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3a450e-1d3c-4ea3-a432-3e57dbeb27b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:34.804693 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3a450e-1d3c-4ea3-a432-3e57dbeb27b3 None admin] RBAC: Adding query filter params (role.id=305a1ca2e6f740eba7dc696849e933eb, scope.project.id=3094ff8966f149bda784da6a51b1358f, user.id=e44f6cd435be4e03bc79cb3125d71451) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:34.805008 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3a450e-1d3c-4ea3-a432-3e57dbeb27b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:34.806446 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3a450e-1d3c-4ea3-a432-3e57dbeb27b3 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:34.824118 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 267/536] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jun 15 14:31:34 2022] GET /identity/v3/role_assignments?user.id=e44f6cd435be4e03bc79cb3125d71451&scope.project.id=3094ff8966f149bda784da6a51b1358f&role.id=305a1ca2e6f740eba7dc696849e933eb => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:34.824118 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3a450e-1d3c-4ea3-a432-3e57dbeb27b3 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:34.885104 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed534a0-3a4a-4859-b65f-9f608e77ebdd None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:34.935345 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9d7cfc-fb3d-4792-bb1e-84b6d265a01f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:34.936431 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9d7cfc-fb3d-4792-bb1e-84b6d265a01f None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:34.936782 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9d7cfc-fb3d-4792-bb1e-84b6d265a01f None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:34.937368 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9d7cfc-fb3d-4792-bb1e-84b6d265a01f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:34.938762 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9d7cfc-fb3d-4792-bb1e-84b6d265a01f None admin] RBAC: Adding query filter params (role.id=305a1ca2e6f740eba7dc696849e933eb, scope.project.id=3094ff8966f149bda784da6a51b1358f, user.id=4f18c57f27784694935bfb0bcea48a78) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:34.939349 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9d7cfc-fb3d-4792-bb1e-84b6d265a01f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:34.941641 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9d7cfc-fb3d-4792-bb1e-84b6d265a01f None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:34.948052 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3a450e-1d3c-4ea3-a432-3e57dbeb27b3 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:34.952275 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 270/537] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jun 15 14:31:34 2022] GET /identity/v3/role_assignments?user.id=4f18c57f27784694935bfb0bcea48a78&scope.project.id=3094ff8966f149bda784da6a51b1358f&role.id=305a1ca2e6f740eba7dc696849e933eb => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:34.953794 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a99fd5e-ccef-466f-a4b1-2f241bf49cb2 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:34.954141 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a99fd5e-ccef-466f-a4b1-2f241bf49cb2 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:34.954433 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a99fd5e-ccef-466f-a4b1-2f241bf49cb2 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:34.960040 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 271/538] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:31:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:34.962481 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad3b88b-7e8f-46df-a15e-efd1d03f5288 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:34.963065 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad3b88b-7e8f-46df-a15e-efd1d03f5288 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:34.963640 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad3b88b-7e8f-46df-a15e-efd1d03f5288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:34.984573 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-cad3b88b-7e8f-46df-a15e-efd1d03f5288 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:34.986341 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-cad3b88b-7e8f-46df-a15e-efd1d03f5288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:35.033635 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ea67f-5670-4052-b577-36ea7377e663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:35.035145 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ea67f-5670-4052-b577-36ea7377e663 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:35.035433 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ea67f-5670-4052-b577-36ea7377e663 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:35.035716 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ea67f-5670-4052-b577-36ea7377e663 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:35.036559 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619ea67f-5670-4052-b577-36ea7377e663 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:35.036874 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619ea67f-5670-4052-b577-36ea7377e663 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:35.038285 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619ea67f-5670-4052-b577-36ea7377e663 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:35.058650 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 268/539] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:31:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:35.063324 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ea67f-5670-4052-b577-36ea7377e663 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:35.096619 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 272/540] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:31:35.117307 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:35.157448 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad3b88b-7e8f-46df-a15e-efd1d03f5288 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:35.199691 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ea67f-5670-4052-b577-36ea7377e663 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:35.210154 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:35.252847 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889ba03e-d162-4bc5-9d62-8ae999552fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:35.254007 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889ba03e-d162-4bc5-9d62-8ae999552fb6 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:35.254291 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889ba03e-d162-4bc5-9d62-8ae999552fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:35.254563 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889ba03e-d162-4bc5-9d62-8ae999552fb6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:35.262138 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889ba03e-d162-4bc5-9d62-8ae999552fb6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:35.263567 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889ba03e-d162-4bc5-9d62-8ae999552fb6 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:35.283122 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad3b88b-7e8f-46df-a15e-efd1d03f5288 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:35.290214 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-889ba03e-d162-4bc5-9d62-8ae999552fb6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:31:35.290214 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:35.290214 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:35.290214 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:35.290214 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:35.290214 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:35.290214 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:35.290214 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:35.290214 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:35.290214 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:35.290214 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:35.290214 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:35.290214 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:31:35.290214 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:31:35.290214 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:31:35.291069 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:31:35.291069 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:35.291069 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:35.291069 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:35.291069 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:35.291069 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:35.291069 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:35.291069 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:35.291069 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:35.291069 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:35.291069 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:35.291069 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:35.291069 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:35.291069 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:35.291069 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:35.291069 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:35.291069 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:35.291896 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:31:35.291896 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:31:35.291896 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:31:35.291896 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:31:35.291896 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:31:35.291896 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:31:35.291896 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:31:35.291896 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:35.300534 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 269/541] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jun 15 14:31:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:35.305942 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889ba03e-d162-4bc5-9d62-8ae999552fb6 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:35.323142 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d43cca-c4cf-429e-90d1-44c627d7a69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:35.324287 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d43cca-c4cf-429e-90d1-44c627d7a69d None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:35.326886 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d43cca-c4cf-429e-90d1-44c627d7a69d None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:35.328573 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d43cca-c4cf-429e-90d1-44c627d7a69d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:35.348878 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d43cca-c4cf-429e-90d1-44c627d7a69d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:31:35.358700 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-97d43cca-c4cf-429e-90d1-44c627d7a69d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:31:35.358700 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:35.358700 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:35.358700 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:35.358700 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:35.358700 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:35.358700 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:35.358700 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:35.358700 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:35.358700 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:35.358700 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:35.358700 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:35.358700 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:31:35.358700 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:31:35.358700 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:31:35.358700 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:31:35.359583 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:31:35.359583 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:31:35.359583 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:31:35.359583 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:31:35.359583 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:35.359583 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:35.359583 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:35.359583 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:35.359583 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:35.359583 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:35.359583 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:35.359583 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:35.359583 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:35.359583 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:35.359583 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:35.359583 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:35.360407 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:35.360407 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:35.360407 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:35.360407 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:35.360407 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:31:35.360407 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:31:35.360407 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:31:35.360407 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:31:35.360407 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:31:35.360407 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:31:35.360407 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:31:35.360407 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:35.370886 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 273/542] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Jun 15 14:31:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:31:35.380500 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d43cca-c4cf-429e-90d1-44c627d7a69d None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:35.496621 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:35.506661 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889ba03e-d162-4bc5-9d62-8ae999552fb6 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:35.556842 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:35.561480 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d43cca-c4cf-429e-90d1-44c627d7a69d None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:35.593869 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d579a8b1-baf9-457c-bb1c-6b4941a11a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:35.594453 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d579a8b1-baf9-457c-bb1c-6b4941a11a80 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:35.594739 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d579a8b1-baf9-457c-bb1c-6b4941a11a80 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:35.595017 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d579a8b1-baf9-457c-bb1c-6b4941a11a80 None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:35.595798 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d579a8b1-baf9-457c-bb1c-6b4941a11a80 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:35.596122 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d579a8b1-baf9-457c-bb1c-6b4941a11a80 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:35.600119 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d579a8b1-baf9-457c-bb1c-6b4941a11a80 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:35.604111 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cecc235-3e32-4c78-bbbf-83c74b9cc972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:35.604111 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cecc235-3e32-4c78-bbbf-83c74b9cc972 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:35.604111 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cecc235-3e32-4c78-bbbf-83c74b9cc972 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:35.604111 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cecc235-3e32-4c78-bbbf-83c74b9cc972 None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:35.604430 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cecc235-3e32-4c78-bbbf-83c74b9cc972 None admin] RBAC: Adding query filter params (name=service) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:35.604430 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cecc235-3e32-4c78-bbbf-83c74b9cc972 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:35.605392 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cecc235-3e32-4c78-bbbf-83c74b9cc972 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:35.620637 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 270/543] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jun 15 14:31:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:31:35.628441 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d579a8b1-baf9-457c-bb1c-6b4941a11a80 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:35.631963 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 274/544] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Jun 15 14:31:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:35.632824 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cecc235-3e32-4c78-bbbf-83c74b9cc972 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:35.814043 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d579a8b1-baf9-457c-bb1c-6b4941a11a80 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:35.818715 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cecc235-3e32-4c78-bbbf-83c74b9cc972 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:35.855894 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a2ff3b-c4b3-4787-867c-097045b12215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:35.857427 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a2ff3b-c4b3-4787-867c-097045b12215 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:35.857874 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a2ff3b-c4b3-4787-867c-097045b12215 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:35.858169 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a2ff3b-c4b3-4787-867c-097045b12215 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:35.869399 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a2ff3b-c4b3-4787-867c-097045b12215 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:35.870849 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a2ff3b-c4b3-4787-867c-097045b12215 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:35.889699 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be32817-49f1-427c-a549-c602faa0c827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:35.890211 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-66a2ff3b-c4b3-4787-867c-097045b12215 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:35.890211 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:35.890211 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:35.890211 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:35.890211 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:35.890211 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:35.890211 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:35.890211 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:35.890211 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:35.890211 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:35.890211 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:35.890211 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:35.890211 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:31:35.890211 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:31:35.890211 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:31:35.890980 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:31:35.890980 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:35.890980 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:35.890980 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:35.890980 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:35.890980 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:35.890980 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:35.890980 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:35.890980 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:35.890980 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:35.890980 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:35.890980 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:35.890980 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:35.890980 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:35.890980 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:35.890980 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:35.890980 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:35.891816 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:31:35.891816 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:31:35.891816 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:35.891816 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:35.893168 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be32817-49f1-427c-a549-c602faa0c827 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:35.893644 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be32817-49f1-427c-a549-c602faa0c827 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:35.894203 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be32817-49f1-427c-a549-c602faa0c827 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:35.895217 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be32817-49f1-427c-a549-c602faa0c827 None admin] RBAC: Adding query filter params (role.id=305a1ca2e6f740eba7dc696849e933eb, scope.project.id=3094ff8966f149bda784da6a51b1358f, user.id=c739b61c8a8b453e979bda720bafebb1) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:35.895588 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 275/545] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:31:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:35.895937 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be32817-49f1-427c-a549-c602faa0c827 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:35.896782 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a2ff3b-c4b3-4787-867c-097045b12215 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:35.897776 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be32817-49f1-427c-a549-c602faa0c827 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:35.912406 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 271/546] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jun 15 14:31:35 2022] GET /identity/v3/role_assignments?user.id=c739b61c8a8b453e979bda720bafebb1&scope.project.id=3094ff8966f149bda784da6a51b1358f&role.id=305a1ca2e6f740eba7dc696849e933eb => generated 259 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:35.913647 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:36.056590 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a2ff3b-c4b3-4787-867c-097045b12215 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:36.103681 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c13f4e-7276-4658-a924-d6938c389537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:36.104780 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c13f4e-7276-4658-a924-d6938c389537 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:36.105259 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c13f4e-7276-4658-a924-d6938c389537 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:36.105729 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c13f4e-7276-4658-a924-d6938c389537 None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:36.107056 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c13f4e-7276-4658-a924-d6938c389537 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:36.107302 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c13f4e-7276-4658-a924-d6938c389537 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:36.108708 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c13f4e-7276-4658-a924-d6938c389537 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:36.121442 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 276/547] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:31:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:36.127755 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be32817-49f1-427c-a549-c602faa0c827 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:36.156986 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:36.237515 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be32817-49f1-427c-a549-c602faa0c827 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:36.272020 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ddc2f7-97e2-4fc9-9fa8-eae3d6a23559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:36.272842 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ddc2f7-97e2-4fc9-9fa8-eae3d6a23559 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:36.273128 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ddc2f7-97e2-4fc9-9fa8-eae3d6a23559 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:36.273403 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ddc2f7-97e2-4fc9-9fa8-eae3d6a23559 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:36.282350 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ddc2f7-97e2-4fc9-9fa8-eae3d6a23559 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:36.283815 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ddc2f7-97e2-4fc9-9fa8-eae3d6a23559 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:36.298777 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-d4ddc2f7-97e2-4fc9-9fa8-eae3d6a23559 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 15 14:31:36.298777 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:36.298777 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:36.298777 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:36.298777 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:36.298777 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:36.298777 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:36.298777 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:36.298777 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:36.298777 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:36.298777 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:36.298777 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:36.298777 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:31:36.298777 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:31:36.298777 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:31:36.298777 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:31:36.299925 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:36.299925 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:36.299925 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:31:36.299925 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:31:36.299925 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:31:36.299925 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:31:36.299925 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:36.299925 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:36.299925 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:36.299925 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:36.299925 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:36.299925 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:36.299925 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:36.299925 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:36.299925 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:36.299925 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:36.299925 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:36.299925 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:36.300868 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:36.300868 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:36.300868 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:31:36.300868 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:31:36.300868 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:31:36.300868 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:31:36.300868 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:31:36.300868 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:31:36.300868 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 15 14:31:36.300868 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:36.300868 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 272/548] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jun 15 14:31:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:36.309578 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c13f4e-7276-4658-a924-d6938c389537 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:36.381296 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:36.428524 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c13f4e-7276-4658-a924-d6938c389537 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:36.479171 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ee0bfc-b37b-480e-97b7-6f2c9ca5ae88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:36.480235 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ee0bfc-b37b-480e-97b7-6f2c9ca5ae88 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:36.480726 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ee0bfc-b37b-480e-97b7-6f2c9ca5ae88 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:36.481948 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ee0bfc-b37b-480e-97b7-6f2c9ca5ae88 None admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:36.482996 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ee0bfc-b37b-480e-97b7-6f2c9ca5ae88 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:36.483512 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ee0bfc-b37b-480e-97b7-6f2c9ca5ae88 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:36.485195 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ee0bfc-b37b-480e-97b7-6f2c9ca5ae88 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:36.504647 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c025594-2615-4f78-a206-c94e540da775 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:36.504968 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c025594-2615-4f78-a206-c94e540da775 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:36.505202 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c025594-2615-4f78-a206-c94e540da775 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:36.506767 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 273/549] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:31:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:36.512137 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da92cd8c-16e2-4685-ae85-f35ef89874d0 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:36.512137 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da92cd8c-16e2-4685-ae85-f35ef89874d0 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:36.512137 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da92cd8c-16e2-4685-ae85-f35ef89874d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:36.516948 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 277/550] 10.222.0.42 () {56 vars in 1135 bytes} [Wed Jun 15 14:31:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:36.522212 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ee0bfc-b37b-480e-97b7-6f2c9ca5ae88 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:36.530953 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-da92cd8c-16e2-4685-ae85-f35ef89874d0 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:36.532174 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-da92cd8c-16e2-4685-ae85-f35ef89874d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:36.625398 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 274/551] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:31:36.669277 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da92cd8c-16e2-4685-ae85-f35ef89874d0 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:36.694538 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ee0bfc-b37b-480e-97b7-6f2c9ca5ae88 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:36.697853 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:36.729681 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:36.736688 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19db0c35-20d3-40f3-b092-1450643bbb2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:36.737408 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db0c35-20d3-40f3-b092-1450643bbb2c None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:36.737690 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db0c35-20d3-40f3-b092-1450643bbb2c None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:36.737969 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db0c35-20d3-40f3-b092-1450643bbb2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:36.812640 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19db0c35-20d3-40f3-b092-1450643bbb2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:36.814102 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19db0c35-20d3-40f3-b092-1450643bbb2c None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:36.950005 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da92cd8c-16e2-4685-ae85-f35ef89874d0 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:36.955014 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 278/552] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jun 15 14:31:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 15 14:31:36.962085 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19db0c35-20d3-40f3-b092-1450643bbb2c None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:37.020475 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7191c7c8-f214-4e63-b616-44694bdc295e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:37.021661 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7191c7c8-f214-4e63-b616-44694bdc295e None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:37.022163 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7191c7c8-f214-4e63-b616-44694bdc295e None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:37.022646 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7191c7c8-f214-4e63-b616-44694bdc295e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:37.028982 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:37.051250 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7191c7c8-f214-4e63-b616-44694bdc295e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:31:37.065652 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-7191c7c8-f214-4e63-b616-44694bdc295e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:31:37.065652 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:37.065652 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:37.065652 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:37.065652 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:37.065652 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:37.065652 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:37.065652 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:37.065652 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:37.065652 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:37.065652 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:37.065652 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:37.065652 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:31:37.065652 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:31:37.065652 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:31:37.065652 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:31:37.066579 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:31:37.066579 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:31:37.066579 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:31:37.066579 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:31:37.066579 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:37.066579 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:37.066579 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:37.066579 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:37.066579 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:37.066579 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:37.066579 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:37.066579 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:37.066579 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:37.066579 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:37.066579 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:37.066579 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:37.067396 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:37.067396 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:37.067396 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:37.067396 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:37.067396 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:31:37.067396 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:31:37.067396 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:31:37.067396 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:31:37.067396 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:31:37.067396 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:31:37.067396 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:31:37.067396 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:37.072460 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 275/553] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Jun 15 14:31:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 404 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:37.079282 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7191c7c8-f214-4e63-b616-44694bdc295e None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:37.142326 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19db0c35-20d3-40f3-b092-1450643bbb2c None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:37.189179 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8679c2c7-f24a-4c3d-9b29-0ab4ac07dbe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:37.191294 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8679c2c7-f24a-4c3d-9b29-0ab4ac07dbe6 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:37.192641 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8679c2c7-f24a-4c3d-9b29-0ab4ac07dbe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:37.193197 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8679c2c7-f24a-4c3d-9b29-0ab4ac07dbe6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:37.214262 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:37.218310 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8679c2c7-f24a-4c3d-9b29-0ab4ac07dbe6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:37.220104 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8679c2c7-f24a-4c3d-9b29-0ab4ac07dbe6 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:37.240657 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-8679c2c7-f24a-4c3d-9b29-0ab4ac07dbe6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:37.240657 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:37.240657 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:37.240657 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:37.240657 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:37.240657 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:37.240657 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:37.240657 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:37.240657 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:37.240657 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:37.240657 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:37.240657 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:37.240657 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:31:37.240657 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:31:37.240657 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:31:37.241564 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:31:37.241564 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:37.241564 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:37.241564 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:37.241564 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:37.241564 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:37.241564 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:37.241564 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:37.241564 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:37.241564 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:37.241564 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:37.241564 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:37.241564 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:37.241564 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:37.241564 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:37.241564 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:37.241564 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:37.242448 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:31:37.242448 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:31:37.242448 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:37.242448 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:37.246239 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 279/554] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:31:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:31:37.252304 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8679c2c7-f24a-4c3d-9b29-0ab4ac07dbe6 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:37.281922 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7191c7c8-f214-4e63-b616-44694bdc295e None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:37.328127 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f6c21d-23bb-4095-ad3e-7764a44ca21b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:37.328825 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f6c21d-23bb-4095-ad3e-7764a44ca21b None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:37.329110 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f6c21d-23bb-4095-ad3e-7764a44ca21b None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:37.329389 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f6c21d-23bb-4095-ad3e-7764a44ca21b None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:37.330154 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f6c21d-23bb-4095-ad3e-7764a44ca21b None admin] RBAC: Adding query filter params (name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:37.330467 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f6c21d-23bb-4095-ad3e-7764a44ca21b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:37.332047 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f6c21d-23bb-4095-ad3e-7764a44ca21b None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:37.342734 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 276/555] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Jun 15 14:31:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:37.349220 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f6c21d-23bb-4095-ad3e-7764a44ca21b None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:37.400834 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:37.443742 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8679c2c7-f24a-4c3d-9b29-0ab4ac07dbe6 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:37.493314 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc55b90e-525e-49d3-82e8-f89b8bbea1c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:37.494092 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc55b90e-525e-49d3-82e8-f89b8bbea1c9 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:37.494567 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc55b90e-525e-49d3-82e8-f89b8bbea1c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:37.494937 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc55b90e-525e-49d3-82e8-f89b8bbea1c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:37.495663 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc55b90e-525e-49d3-82e8-f89b8bbea1c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:37.496068 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc55b90e-525e-49d3-82e8-f89b8bbea1c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:37.497391 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f6c21d-23bb-4095-ad3e-7764a44ca21b None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:37.497779 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc55b90e-525e-49d3-82e8-f89b8bbea1c9 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:37.510406 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 280/556] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:31:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:31:37.511937 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef23f9e-4240-4247-a8d5-448db087130e None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:37.512882 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef23f9e-4240-4247-a8d5-448db087130e None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:37.513494 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef23f9e-4240-4247-a8d5-448db087130e None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:37.518987 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 281/557] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:31:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:37.518987 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef23f9e-4240-4247-a8d5-448db087130e None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:37.535915 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:37.545781 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c99d055-763b-442e-9d41-3781b7bd0200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:37.547415 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c99d055-763b-442e-9d41-3781b7bd0200 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:37.547892 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c99d055-763b-442e-9d41-3781b7bd0200 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:37.554653 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c99d055-763b-442e-9d41-3781b7bd0200 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:37.568004 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c99d055-763b-442e-9d41-3781b7bd0200 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:37.569898 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c99d055-763b-442e-9d41-3781b7bd0200 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:37.591427 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-2c99d055-763b-442e-9d41-3781b7bd0200 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:37.591427 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:37.591427 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:37.591427 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:37.591427 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:37.591427 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:37.591427 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:37.591427 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:37.591427 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:37.591427 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:37.591427 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:37.591427 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:37.591427 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:31:37.591427 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:31:37.591427 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:31:37.592872 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:31:37.592872 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:37.592872 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:37.592872 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:37.592872 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:37.592872 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:37.592872 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:37.592872 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:37.592872 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:37.592872 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:37.592872 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:37.592872 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:37.592872 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:37.592872 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:37.592872 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:37.592872 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:37.592872 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:37.593756 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:31:37.593756 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:31:37.593756 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:37.593756 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:37.595838 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 277/558] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:31:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:37.597217 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191fc75e-cf59-43e9-bb87-4f72f95a2001 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:37.597651 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191fc75e-cf59-43e9-bb87-4f72f95a2001 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:37.598053 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191fc75e-cf59-43e9-bb87-4f72f95a2001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:37.615263 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-191fc75e-cf59-43e9-bb87-4f72f95a2001 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:37.616495 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-191fc75e-cf59-43e9-bb87-4f72f95a2001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:37.635545 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef23f9e-4240-4247-a8d5-448db087130e None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:37.670605 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd7ab17-dd71-4b6c-b221-9adb7c5b68b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:37.671394 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd7ab17-dd71-4b6c-b221-9adb7c5b68b9 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:37.671678 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd7ab17-dd71-4b6c-b221-9adb7c5b68b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:37.671954 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd7ab17-dd71-4b6c-b221-9adb7c5b68b9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:37.678903 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd7ab17-dd71-4b6c-b221-9adb7c5b68b9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:37.680486 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd7ab17-dd71-4b6c-b221-9adb7c5b68b9 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:37.695620 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-9fd7ab17-dd71-4b6c-b221-9adb7c5b68b9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:31:37.695620 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:37.695620 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:37.695620 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:37.695620 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:37.695620 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:37.695620 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:37.695620 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:37.695620 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:37.695620 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:37.695620 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:37.695620 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:37.695620 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:31:37.695620 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:31:37.695620 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:31:37.696739 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:31:37.696739 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:37.696739 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:37.696739 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:37.696739 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:37.696739 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:37.696739 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:37.696739 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:37.696739 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:37.696739 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:37.696739 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:37.696739 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:37.696739 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:37.696739 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:37.696739 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:37.696739 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:37.696739 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:37.697711 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:31:37.697711 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:31:37.697711 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:31:37.697711 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:31:37.697711 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:31:37.697711 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:31:37.697711 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:31:37.697711 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:37.698638 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 282/559] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jun 15 14:31:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:37.699674 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd7ab17-dd71-4b6c-b221-9adb7c5b68b9 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:37.700902 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 278/560] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:31:37.707890 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a9eda0-f987-4dfb-94b3-40ed284b1f06 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:37.707890 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a9eda0-f987-4dfb-94b3-40ed284b1f06 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:37.707890 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a9eda0-f987-4dfb-94b3-40ed284b1f06 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:37.707890 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 279/561] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:31:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:37.707890 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a9eda0-f987-4dfb-94b3-40ed284b1f06 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:37.716235 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:37.813233 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a9eda0-f987-4dfb-94b3-40ed284b1f06 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:37.816915 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd7ab17-dd71-4b6c-b221-9adb7c5b68b9 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:37.828113 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:37.852380 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa183f49-61d3-4e6e-a99f-1bf5e06402e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:37.853314 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa183f49-61d3-4e6e-a99f-1bf5e06402e5 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:37.853781 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa183f49-61d3-4e6e-a99f-1bf5e06402e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:37.854260 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa183f49-61d3-4e6e-a99f-1bf5e06402e5 None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:37.855223 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa183f49-61d3-4e6e-a99f-1bf5e06402e5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:37.855714 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa183f49-61d3-4e6e-a99f-1bf5e06402e5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:37.855930 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fb96fd-faa5-48e0-8bf2-ba08a89dfc31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:37.857137 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb96fd-faa5-48e0-8bf2-ba08a89dfc31 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:37.857414 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb96fd-faa5-48e0-8bf2-ba08a89dfc31 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:37.857691 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb96fd-faa5-48e0-8bf2-ba08a89dfc31 None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:37.858407 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fb96fd-faa5-48e0-8bf2-ba08a89dfc31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:37.858711 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fb96fd-faa5-48e0-8bf2-ba08a89dfc31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:37.859453 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa183f49-61d3-4e6e-a99f-1bf5e06402e5 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:37.860115 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fb96fd-faa5-48e0-8bf2-ba08a89dfc31 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:37.873494 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 283/562] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:31:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:37.874385 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 280/563] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jun 15 14:31:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:37.874737 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73812f12-770e-4636-b0c1-54408e305cd1 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:37.875028 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73812f12-770e-4636-b0c1-54408e305cd1 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:37.875308 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73812f12-770e-4636-b0c1-54408e305cd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:37.876004 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa183f49-61d3-4e6e-a99f-1bf5e06402e5 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:37.897347 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-73812f12-770e-4636-b0c1-54408e305cd1 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:37.898529 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-73812f12-770e-4636-b0c1-54408e305cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:37.965201 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:37.976736 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 284/564] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:31:37.977568 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73812f12-770e-4636-b0c1-54408e305cd1 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:37.998335 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa183f49-61d3-4e6e-a99f-1bf5e06402e5 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:38.037420 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08520ab3-63e8-4a99-94f0-c6e7a52e6d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:38.038557 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08520ab3-63e8-4a99-94f0-c6e7a52e6d51 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:38.039022 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08520ab3-63e8-4a99-94f0-c6e7a52e6d51 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:38.039507 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08520ab3-63e8-4a99-94f0-c6e7a52e6d51 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:38.052669 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08520ab3-63e8-4a99-94f0-c6e7a52e6d51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:31:38.061194 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-08520ab3-63e8-4a99-94f0-c6e7a52e6d51 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:31:38.061194 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:38.061194 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:38.061194 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:38.061194 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:38.061194 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:38.061194 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:38.061194 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:38.061194 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:38.061194 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:38.061194 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:38.061194 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:38.061194 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:31:38.061194 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:31:38.061194 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:31:38.061194 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:31:38.062169 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:31:38.062169 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:31:38.062169 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:31:38.062169 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:31:38.062169 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:38.062169 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:38.062169 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:38.062169 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:38.062169 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:38.062169 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:38.062169 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:38.062169 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:38.062169 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:38.062169 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:38.062169 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:38.062169 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:38.063034 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:38.063034 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:38.063034 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:38.063034 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:38.063034 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:31:38.063034 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:31:38.063034 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:31:38.063034 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:31:38.063034 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:31:38.063034 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:31:38.063034 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:31:38.063034 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:38.067376 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 281/565] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Jun 15 14:31:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:38.068212 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08520ab3-63e8-4a99-94f0-c6e7a52e6d51 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:38.095723 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73812f12-770e-4636-b0c1-54408e305cd1 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:38.103366 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:38.133180 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55792a1c-ccba-4206-9c16-22a4ba0ad231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:38.134140 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55792a1c-ccba-4206-9c16-22a4ba0ad231 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:38.134493 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55792a1c-ccba-4206-9c16-22a4ba0ad231 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:38.134777 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55792a1c-ccba-4206-9c16-22a4ba0ad231 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:38.143211 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55792a1c-ccba-4206-9c16-22a4ba0ad231 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:38.144704 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55792a1c-ccba-4206-9c16-22a4ba0ad231 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:38.160066 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-55792a1c-ccba-4206-9c16-22a4ba0ad231 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:31:38.160066 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:38.160066 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:38.160066 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:38.160066 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:38.160066 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:38.160066 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:38.160066 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:38.160066 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:38.160066 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:38.160066 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:38.160066 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:38.160066 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:31:38.160066 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:31:38.160066 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:31:38.160066 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:31:38.161102 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:38.161102 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:38.161102 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:31:38.161102 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:31:38.161102 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:31:38.161102 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:31:38.161102 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:38.161102 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:38.161102 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:38.161102 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:38.161102 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:38.161102 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:38.161102 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:38.161102 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:38.161102 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:38.161102 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:38.161102 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:38.161102 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:38.162062 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:38.162062 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:38.162062 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:31:38.162062 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:31:38.162062 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:31:38.162062 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:31:38.162062 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:31:38.162062 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:31:38.162062 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:31:38.162062 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:38.162062 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 285/566] 10.222.0.42 () {56 vars in 1093 bytes} [Wed Jun 15 14:31:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:38.162062 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55792a1c-ccba-4206-9c16-22a4ba0ad231 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:38.182197 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08520ab3-63e8-4a99-94f0-c6e7a52e6d51 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:38.214914 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:38.219719 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04d66c7-b90e-432d-9504-15331151b498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:38.220408 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d66c7-b90e-432d-9504-15331151b498 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:38.220696 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d66c7-b90e-432d-9504-15331151b498 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:38.220971 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d66c7-b90e-432d-9504-15331151b498 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:38.221796 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04d66c7-b90e-432d-9504-15331151b498 None admin] RBAC: Adding query filter params (role.id=305a1ca2e6f740eba7dc696849e933eb, scope.project.id=3094ff8966f149bda784da6a51b1358f, user.id=c04dc5832333498e96a7e02bbfb9484c) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:38.222107 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04d66c7-b90e-432d-9504-15331151b498 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:38.223526 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04d66c7-b90e-432d-9504-15331151b498 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:38.229214 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 282/567] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jun 15 14:31:38 2022] GET /identity/v3/role_assignments?user.id=c04dc5832333498e96a7e02bbfb9484c&scope.project.id=3094ff8966f149bda784da6a51b1358f&role.id=305a1ca2e6f740eba7dc696849e933eb => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:38.229996 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04d66c7-b90e-432d-9504-15331151b498 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:38.281863 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55792a1c-ccba-4206-9c16-22a4ba0ad231 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:38.325470 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6baed35-b6f3-4892-96e1-9dcdea950d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:38.326864 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6baed35-b6f3-4892-96e1-9dcdea950d94 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:38.327404 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6baed35-b6f3-4892-96e1-9dcdea950d94 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:38.328027 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6baed35-b6f3-4892-96e1-9dcdea950d94 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:38.343282 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6baed35-b6f3-4892-96e1-9dcdea950d94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:31:38.354956 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:38.357041 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-b6baed35-b6f3-4892-96e1-9dcdea950d94 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:31:38.357041 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:38.357041 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:38.357041 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:38.357041 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:38.357041 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:38.357041 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:38.357041 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:38.357041 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:38.357041 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:38.357041 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:38.357041 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:38.357041 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:31:38.357041 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:31:38.357041 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:31:38.357041 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:31:38.357991 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:31:38.357991 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:31:38.357991 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:31:38.357991 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:31:38.357991 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:38.357991 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:38.357991 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:38.357991 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:38.357991 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:38.357991 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:38.357991 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:38.357991 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:38.357991 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:38.357991 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:38.357991 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:38.357991 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:38.369350 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:38.369350 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:38.369350 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:38.369350 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:38.369350 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:31:38.369350 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:31:38.369350 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:31:38.369350 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:31:38.369350 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:31:38.369350 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:31:38.369350 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:31:38.369350 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:38.369350 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 286/568] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Jun 15 14:31:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:38.369350 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6baed35-b6f3-4892-96e1-9dcdea950d94 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:38.370110 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04d66c7-b90e-432d-9504-15331151b498 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:38.406987 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3012df-4e77-4675-af0c-1b6597006e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:38.407697 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3012df-4e77-4675-af0c-1b6597006e4d None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:38.407964 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3012df-4e77-4675-af0c-1b6597006e4d None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:38.408276 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3012df-4e77-4675-af0c-1b6597006e4d None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:38.409065 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3012df-4e77-4675-af0c-1b6597006e4d None admin] RBAC: Adding query filter params (name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:38.409353 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3012df-4e77-4675-af0c-1b6597006e4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:38.410750 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3012df-4e77-4675-af0c-1b6597006e4d None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:38.424355 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 283/569] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Jun 15 14:31:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:38.424355 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3012df-4e77-4675-af0c-1b6597006e4d None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:38.489497 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6baed35-b6f3-4892-96e1-9dcdea950d94 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:38.503070 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:38.527792 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de40deb7-b4eb-46d9-b8bb-fee97b1cc332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:38.529279 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de40deb7-b4eb-46d9-b8bb-fee97b1cc332 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:38.529946 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de40deb7-b4eb-46d9-b8bb-fee97b1cc332 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:38.530609 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de40deb7-b4eb-46d9-b8bb-fee97b1cc332 None admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:38.530852 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3012df-4e77-4675-af0c-1b6597006e4d None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:38.531953 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de40deb7-b4eb-46d9-b8bb-fee97b1cc332 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:38.532630 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de40deb7-b4eb-46d9-b8bb-fee97b1cc332 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:38.535050 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de40deb7-b4eb-46d9-b8bb-fee97b1cc332 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:38.559284 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 287/570] 10.222.0.42 () {56 vars in 1133 bytes} [Wed Jun 15 14:31:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:38.560743 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eae10a9-3675-4ff9-9154-5449747bf606 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:38.561074 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eae10a9-3675-4ff9-9154-5449747bf606 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:38.561364 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eae10a9-3675-4ff9-9154-5449747bf606 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:38.563069 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 288/571] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:31:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:38.563605 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eae10a9-3675-4ff9-9154-5449747bf606 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:38.571917 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11630644-3486-4a1b-ab25-e647d9a0637e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:38.572872 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11630644-3486-4a1b-ab25-e647d9a0637e None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:38.573325 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11630644-3486-4a1b-ab25-e647d9a0637e None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:38.573772 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11630644-3486-4a1b-ab25-e647d9a0637e None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:38.574588 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11630644-3486-4a1b-ab25-e647d9a0637e None admin] RBAC: Adding query filter params (name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:38.574955 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11630644-3486-4a1b-ab25-e647d9a0637e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:38.576498 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11630644-3486-4a1b-ab25-e647d9a0637e None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:38.589813 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 284/572] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Jun 15 14:31:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:38.591388 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11630644-3486-4a1b-ab25-e647d9a0637e None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:38.616811 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:38.672722 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eae10a9-3675-4ff9-9154-5449747bf606 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:38.698869 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11630644-3486-4a1b-ab25-e647d9a0637e None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:38.710441 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2c7cc5-eba5-424e-8ee4-818859b54747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:38.711327 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2c7cc5-eba5-424e-8ee4-818859b54747 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:38.711686 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2c7cc5-eba5-424e-8ee4-818859b54747 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:38.711969 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2c7cc5-eba5-424e-8ee4-818859b54747 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:38.723429 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2c7cc5-eba5-424e-8ee4-818859b54747 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:38.724970 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2c7cc5-eba5-424e-8ee4-818859b54747 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:38.738438 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38c9bbe-4901-44a7-934f-d85dcc620c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:38.739232 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38c9bbe-4901-44a7-934f-d85dcc620c51 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:38.739515 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38c9bbe-4901-44a7-934f-d85dcc620c51 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:38.739794 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38c9bbe-4901-44a7-934f-d85dcc620c51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:38.744967 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-8a2c7cc5-eba5-424e-8ee4-818859b54747 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:38.744967 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:38.744967 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:38.744967 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:38.744967 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:38.744967 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:38.744967 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:38.744967 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:38.744967 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:38.744967 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:38.744967 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:38.744967 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:38.744967 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:31:38.744967 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:31:38.744967 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:31:38.745915 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:31:38.745915 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:38.745915 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:38.745915 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:38.745915 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:38.745915 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:38.745915 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:38.745915 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:38.745915 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:38.745915 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:38.745915 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:38.745915 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:38.745915 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:38.745915 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:38.745915 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:38.745915 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:38.745915 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:38.746994 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:31:38.746994 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:31:38.746994 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:38.746994 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:38.746994 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 289/573] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:31:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:38.747799 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58484b4-2ef0-4524-bffc-1dcbe97aa373 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:38.748114 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58484b4-2ef0-4524-bffc-1dcbe97aa373 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:38.748401 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58484b4-2ef0-4524-bffc-1dcbe97aa373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:38.754992 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38c9bbe-4901-44a7-934f-d85dcc620c51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:38.756516 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38c9bbe-4901-44a7-934f-d85dcc620c51 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:38.768913 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-b58484b4-2ef0-4524-bffc-1dcbe97aa373 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:38.770108 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-b58484b4-2ef0-4524-bffc-1dcbe97aa373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:38.774701 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-e38c9bbe-4901-44a7-934f-d85dcc620c51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:38.774701 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:38.774701 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:38.774701 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:38.774701 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:38.774701 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:38.774701 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:38.774701 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:38.774701 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:38.774701 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:38.774701 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:38.774701 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:38.774701 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:31:38.774701 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:31:38.774701 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:31:38.775568 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:31:38.775568 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:38.775568 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:38.775568 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:38.775568 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:38.775568 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:38.775568 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:38.775568 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:38.775568 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:38.775568 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:38.775568 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:38.775568 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:38.775568 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:38.775568 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:38.775568 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:38.775568 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:38.775568 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:38.776813 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:31:38.776813 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:31:38.776813 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:38.776813 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:38.776813 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 285/574] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:31:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:38.777236 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38c9bbe-4901-44a7-934f-d85dcc620c51 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:38.792890 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:38.855019 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:38.857979 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 290/575] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:31:38.857979 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58484b4-2ef0-4524-bffc-1dcbe97aa373 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:38.886788 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38c9bbe-4901-44a7-934f-d85dcc620c51 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:38.925072 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74bb47e-f72f-4229-bb13-e8976d14b6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:38.926014 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74bb47e-f72f-4229-bb13-e8976d14b6b4 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:38.926392 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74bb47e-f72f-4229-bb13-e8976d14b6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:38.926762 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74bb47e-f72f-4229-bb13-e8976d14b6b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:38.938617 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74bb47e-f72f-4229-bb13-e8976d14b6b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:38.940432 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74bb47e-f72f-4229-bb13-e8976d14b6b4 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:38.959162 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58484b4-2ef0-4524-bffc-1dcbe97aa373 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:38.959741 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-e74bb47e-f72f-4229-bb13-e8976d14b6b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:38.959741 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:38.959741 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:38.959741 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:38.959741 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:38.959741 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:38.959741 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:38.959741 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:38.959741 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:38.959741 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:38.959741 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:38.959741 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:38.959741 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:31:38.959741 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:31:38.959741 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:31:38.960705 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:31:38.960705 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:38.960705 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:38.960705 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:38.960705 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:38.960705 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:38.960705 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:38.960705 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:38.960705 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:38.960705 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:38.960705 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:38.960705 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:38.960705 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:38.960705 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:38.960705 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:38.960705 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:38.960705 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:38.961587 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:31:38.961587 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:31:38.961587 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:38.961587 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:38.961587 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 286/576] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:31:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:38.962284 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74bb47e-f72f-4229-bb13-e8976d14b6b4 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:39.022221 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48283369-ee0a-4944-bc9b-53f3a1279a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:39.023270 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48283369-ee0a-4944-bc9b-53f3a1279a21 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:39.023755 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48283369-ee0a-4944-bc9b-53f3a1279a21 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:39.024515 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48283369-ee0a-4944-bc9b-53f3a1279a21 None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:39.025434 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48283369-ee0a-4944-bc9b-53f3a1279a21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:39.025955 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48283369-ee0a-4944-bc9b-53f3a1279a21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:39.027576 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48283369-ee0a-4944-bc9b-53f3a1279a21 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:39.040793 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 291/577] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:31:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:39.041750 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48283369-ee0a-4944-bc9b-53f3a1279a21 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:39.057718 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:39.090537 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74bb47e-f72f-4229-bb13-e8976d14b6b4 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:39.117876 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:39.126673 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136a7679-62c9-40cf-ad0a-9143308471ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:39.127340 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a7679-62c9-40cf-ad0a-9143308471ef None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:39.127628 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a7679-62c9-40cf-ad0a-9143308471ef None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:39.127951 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a7679-62c9-40cf-ad0a-9143308471ef None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:39.128747 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136a7679-62c9-40cf-ad0a-9143308471ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:39.129060 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136a7679-62c9-40cf-ad0a-9143308471ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:39.130430 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136a7679-62c9-40cf-ad0a-9143308471ef None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:39.146752 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 287/578] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:31:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:39.147474 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136a7679-62c9-40cf-ad0a-9143308471ef None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:39.148569 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48283369-ee0a-4944-bc9b-53f3a1279a21 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:39.187522 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702fc8cc-671b-428c-92a7-6c113b22924c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:39.188335 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702fc8cc-671b-428c-92a7-6c113b22924c None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:39.188620 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702fc8cc-671b-428c-92a7-6c113b22924c None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:39.188896 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702fc8cc-671b-428c-92a7-6c113b22924c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:39.200438 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702fc8cc-671b-428c-92a7-6c113b22924c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:31:39.208366 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-702fc8cc-671b-428c-92a7-6c113b22924c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:31:39.208366 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:39.208366 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:39.208366 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:39.208366 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:39.208366 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:39.208366 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:39.208366 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:39.208366 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:39.208366 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:39.208366 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:39.208366 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:39.208366 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:31:39.208366 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:31:39.208366 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:31:39.208366 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:31:39.209462 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:31:39.209462 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:31:39.209462 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:31:39.209462 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:31:39.209462 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:39.209462 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:39.209462 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:39.209462 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:39.209462 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:39.209462 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:39.209462 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:39.209462 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:39.209462 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:39.209462 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:39.209462 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:39.209462 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:39.210296 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:39.210296 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:39.210296 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:39.210296 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:39.210296 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:31:39.210296 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:31:39.210296 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:31:39.210296 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:31:39.210296 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:31:39.210296 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:31:39.210296 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:31:39.210296 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:39.210296 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 292/579] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Jun 15 14:31:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:39.210296 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702fc8cc-671b-428c-92a7-6c113b22924c None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:39.258549 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136a7679-62c9-40cf-ad0a-9143308471ef None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:39.292276 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:39.296068 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379beb7d-f382-419c-8ec0-639d1735dece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:39.296757 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379beb7d-f382-419c-8ec0-639d1735dece None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:39.297037 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379beb7d-f382-419c-8ec0-639d1735dece None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:39.297312 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379beb7d-f382-419c-8ec0-639d1735dece None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:39.298027 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379beb7d-f382-419c-8ec0-639d1735dece None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:39.298336 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379beb7d-f382-419c-8ec0-639d1735dece None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:39.299740 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379beb7d-f382-419c-8ec0-639d1735dece None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:39.311822 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 288/580] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:31:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:39.312661 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379beb7d-f382-419c-8ec0-639d1735dece None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:39.333087 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702fc8cc-671b-428c-92a7-6c113b22924c None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:39.364430 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:39.378961 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6f050a-e4e4-4600-8a86-52015507c9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:39.379916 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6f050a-e4e4-4600-8a86-52015507c9a7 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:39.380383 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6f050a-e4e4-4600-8a86-52015507c9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:39.380710 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6f050a-e4e4-4600-8a86-52015507c9a7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:39.389037 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6f050a-e4e4-4600-8a86-52015507c9a7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:39.390590 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6f050a-e4e4-4600-8a86-52015507c9a7 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:39.406258 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-6b6f050a-e4e4-4600-8a86-52015507c9a7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 14:31:39.406258 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:39.406258 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:39.406258 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:39.406258 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:39.406258 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:39.406258 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:39.406258 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:39.406258 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:39.406258 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:39.406258 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:39.406258 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:39.406258 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:31:39.406258 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:31:39.406258 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:31:39.406258 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:31:39.407428 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:39.407428 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:39.407428 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:31:39.407428 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:31:39.407428 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:31:39.407428 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:31:39.407428 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:39.407428 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:39.407428 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:39.407428 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:39.407428 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:39.407428 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:39.407428 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:39.407428 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:39.407428 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:39.407428 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:39.407428 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:39.407428 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:39.408492 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:39.408492 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:39.408492 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:31:39.408492 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:31:39.408492 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:31:39.408492 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:31:39.408492 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:31:39.408492 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:31:39.408492 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 14:31:39.408492 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:39.408492 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 293/581] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jun 15 14:31:39 2022] GET /identity/v3/users/cinder => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:39.408492 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6f050a-e4e4-4600-8a86-52015507c9a7 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:39.426206 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379beb7d-f382-419c-8ec0-639d1735dece None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:39.472112 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdef1ee-6585-41c5-b41e-27457bfbb5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:39.473026 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdef1ee-6585-41c5-b41e-27457bfbb5c6 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:39.473369 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdef1ee-6585-41c5-b41e-27457bfbb5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:39.473709 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdef1ee-6585-41c5-b41e-27457bfbb5c6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:39.481828 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdef1ee-6585-41c5-b41e-27457bfbb5c6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:39.483269 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdef1ee-6585-41c5-b41e-27457bfbb5c6 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:39.498520 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-9bdef1ee-6585-41c5-b41e-27457bfbb5c6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:31:39.498520 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:39.498520 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:39.498520 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:39.498520 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:39.498520 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:39.498520 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:39.498520 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:39.498520 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:39.498520 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:39.498520 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:39.498520 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:39.498520 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:31:39.498520 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:31:39.498520 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:31:39.499624 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:31:39.499624 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:39.499624 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:39.499624 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:39.499624 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:39.499624 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:39.499624 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:39.499624 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:39.499624 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:39.499624 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:39.499624 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:39.499624 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:39.499624 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:39.499624 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:39.499624 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:39.499624 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:39.499624 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:39.500797 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:31:39.500797 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:31:39.500797 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:31:39.500797 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:31:39.500797 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:31:39.500797 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:31:39.500797 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:31:39.500797 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:39.507020 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 289/582] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jun 15 14:31:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:39.507020 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdef1ee-6585-41c5-b41e-27457bfbb5c6 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:39.532510 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6f050a-e4e4-4600-8a86-52015507c9a7 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:39.570479 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:39.576202 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676e1da6-3bb0-42fb-a934-a9410ae88a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:39.576875 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e1da6-3bb0-42fb-a934-a9410ae88a9c None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:39.577158 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e1da6-3bb0-42fb-a934-a9410ae88a9c None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:39.577433 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e1da6-3bb0-42fb-a934-a9410ae88a9c None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:39.578230 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676e1da6-3bb0-42fb-a934-a9410ae88a9c None admin] RBAC: Adding query filter params (name=service) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:39.578551 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676e1da6-3bb0-42fb-a934-a9410ae88a9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:39.579898 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676e1da6-3bb0-42fb-a934-a9410ae88a9c None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:39.587810 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 294/583] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Jun 15 14:31:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:39.588653 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676e1da6-3bb0-42fb-a934-a9410ae88a9c None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:39.620912 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdef1ee-6585-41c5-b41e-27457bfbb5c6 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:39.629877 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:39.661876 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b31eb4-4372-476b-9ae5-ff9285969445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:39.662753 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b31eb4-4372-476b-9ae5-ff9285969445 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:39.663100 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b31eb4-4372-476b-9ae5-ff9285969445 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:39.663438 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b31eb4-4372-476b-9ae5-ff9285969445 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:39.671878 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b31eb4-4372-476b-9ae5-ff9285969445 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:39.673422 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b31eb4-4372-476b-9ae5-ff9285969445 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:39.688728 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-71b31eb4-4372-476b-9ae5-ff9285969445 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 15 14:31:39.688728 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:39.688728 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:39.688728 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:39.688728 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:39.688728 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:39.688728 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:39.688728 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:39.688728 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:39.688728 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:39.688728 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:39.688728 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:39.688728 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:31:39.688728 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:31:39.688728 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:31:39.688728 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:31:39.689692 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:39.689692 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:39.689692 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:31:39.689692 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:31:39.689692 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:31:39.689692 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:31:39.689692 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:39.689692 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:39.689692 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:39.689692 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:39.689692 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:39.689692 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:39.689692 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:39.689692 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:39.689692 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:39.689692 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:39.689692 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:39.689692 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:39.690599 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:39.690599 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:39.690599 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:31:39.690599 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:31:39.690599 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:31:39.690599 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:31:39.690599 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:31:39.690599 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:31:39.690599 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 15 14:31:39.690599 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:39.690599 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 290/584] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jun 15 14:31:39 2022] GET /identity/v3/users/glance => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:39.691180 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b31eb4-4372-476b-9ae5-ff9285969445 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:39.704676 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676e1da6-3bb0-42fb-a934-a9410ae88a9c None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:39.739147 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c083c186-0adf-460a-9ba1-7f2a0add957f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:39.739814 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c083c186-0adf-460a-9ba1-7f2a0add957f None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:39.740116 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c083c186-0adf-460a-9ba1-7f2a0add957f None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:39.740411 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c083c186-0adf-460a-9ba1-7f2a0add957f None admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:39.741237 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c083c186-0adf-460a-9ba1-7f2a0add957f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:39.741554 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c083c186-0adf-460a-9ba1-7f2a0add957f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:39.742969 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c083c186-0adf-460a-9ba1-7f2a0add957f None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:39.760438 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 295/585] 10.222.0.42 () {56 vars in 1137 bytes} [Wed Jun 15 14:31:39 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:39.761237 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c083c186-0adf-460a-9ba1-7f2a0add957f None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:39.803538 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b31eb4-4372-476b-9ae5-ff9285969445 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:39.821436 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:39.838362 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4d5112-cd19-454e-b5aa-332649886042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:39.839028 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d5112-cd19-454e-b5aa-332649886042 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:39.841009 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d5112-cd19-454e-b5aa-332649886042 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:39.841009 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d5112-cd19-454e-b5aa-332649886042 None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:39.841182 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4d5112-cd19-454e-b5aa-332649886042 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:39.841512 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4d5112-cd19-454e-b5aa-332649886042 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:39.842937 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4d5112-cd19-454e-b5aa-332649886042 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:39.861901 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 291/586] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jun 15 14:31:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:39.861901 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4d5112-cd19-454e-b5aa-332649886042 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:39.881670 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c083c186-0adf-460a-9ba1-7f2a0add957f None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:39.885275 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:39.921057 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b650cc-5acd-4bc1-ad15-22416a702ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:39.922083 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b650cc-5acd-4bc1-ad15-22416a702ba1 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:39.922586 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b650cc-5acd-4bc1-ad15-22416a702ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:39.923131 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b650cc-5acd-4bc1-ad15-22416a702ba1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:39.933652 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b650cc-5acd-4bc1-ad15-22416a702ba1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:39.935360 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b650cc-5acd-4bc1-ad15-22416a702ba1 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:39.954006 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-19b650cc-5acd-4bc1-ad15-22416a702ba1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:39.954006 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:39.954006 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:39.954006 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:39.954006 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:39.954006 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:39.954006 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:39.954006 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:39.954006 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:39.954006 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:39.954006 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:39.954006 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:39.954006 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:31:39.954006 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:31:39.954006 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:31:39.954937 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:31:39.954937 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:39.954937 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:39.954937 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:39.954937 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:39.954937 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:39.954937 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:39.954937 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:39.954937 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:39.954937 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:39.954937 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:39.954937 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:39.954937 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:39.954937 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:39.954937 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:39.954937 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:39.954937 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:39.955833 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:31:39.955833 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:31:39.955833 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:39.955833 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:39.955833 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 296/587] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:31:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:39.956773 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b650cc-5acd-4bc1-ad15-22416a702ba1 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:39.969966 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4d5112-cd19-454e-b5aa-332649886042 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:40.007221 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0d8171-c9a4-4a41-b18b-ab6ab0efe481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:40.007891 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0d8171-c9a4-4a41-b18b-ab6ab0efe481 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:40.008202 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0d8171-c9a4-4a41-b18b-ab6ab0efe481 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:40.008477 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0d8171-c9a4-4a41-b18b-ab6ab0efe481 None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:40.009302 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0d8171-c9a4-4a41-b18b-ab6ab0efe481 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:40.009608 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0d8171-c9a4-4a41-b18b-ab6ab0efe481 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:40.011007 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0d8171-c9a4-4a41-b18b-ab6ab0efe481 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:40.033282 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 292/588] 10.222.0.42 () {56 vars in 1137 bytes} [Wed Jun 15 14:31:39 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:40.033282 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0d8171-c9a4-4a41-b18b-ab6ab0efe481 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:40.068964 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b650cc-5acd-4bc1-ad15-22416a702ba1 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:40.072367 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:40.106738 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2fea7d-5dd6-45a1-a7f9-cad2f760ecf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:40.107527 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2fea7d-5dd6-45a1-a7f9-cad2f760ecf8 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:40.107806 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2fea7d-5dd6-45a1-a7f9-cad2f760ecf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:40.108081 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2fea7d-5dd6-45a1-a7f9-cad2f760ecf8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:40.118928 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2fea7d-5dd6-45a1-a7f9-cad2f760ecf8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:40.120246 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:40.120685 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2fea7d-5dd6-45a1-a7f9-cad2f760ecf8 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:40.136267 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0d8171-c9a4-4a41-b18b-ab6ab0efe481 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:40.138645 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-1d2fea7d-5dd6-45a1-a7f9-cad2f760ecf8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:40.138645 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:40.138645 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:40.138645 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:40.138645 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:40.138645 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:40.138645 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:40.138645 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:40.138645 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:40.138645 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:40.138645 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:40.138645 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:40.138645 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:31:40.138645 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:31:40.138645 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:31:40.139725 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:31:40.139725 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:40.139725 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:40.139725 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:40.139725 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:40.139725 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:40.139725 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:40.139725 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:40.139725 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:40.139725 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:40.139725 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:40.139725 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:40.139725 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:40.139725 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:40.139725 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:40.139725 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:40.139725 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:40.141001 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:31:40.141001 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:31:40.141001 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:40.141001 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:40.144390 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 297/589] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:31:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:40.145213 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2fea7d-5dd6-45a1-a7f9-cad2f760ecf8 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:40.175350 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2daba6-482f-4792-86e9-f0be586daceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:40.176268 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2daba6-482f-4792-86e9-f0be586daceb None admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:40.176547 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2daba6-482f-4792-86e9-f0be586daceb None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:40.176827 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2daba6-482f-4792-86e9-f0be586daceb None admin] PATH_INFO: `/v3/projects/3094ff8966f149bda784da6a51b1358f/users/0927c05c340c48e48b18754888aa9c35/roles/305a1ca2e6f740eba7dc696849e933eb` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:40.191720 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2daba6-482f-4792-86e9-f0be586daceb None admin] RBAC: Authorizing `identity:create_grant(project_id=3094ff8966f149bda784da6a51b1358f, user_id=0927c05c340c48e48b18754888aa9c35, role_id=305a1ca2e6f740eba7dc696849e933eb)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:40.193184 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2daba6-482f-4792-86e9-f0be586daceb None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:40.249440 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2fea7d-5dd6-45a1-a7f9-cad2f760ecf8 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:40.260373 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 293/590] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Jun 15 14:31:40 2022] PUT /identity/v3/projects/3094ff8966f149bda784da6a51b1358f/users/0927c05c340c48e48b18754888aa9c35/roles/305a1ca2e6f740eba7dc696849e933eb => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:31:40.261396 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2daba6-482f-4792-86e9-f0be586daceb None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:40.300992 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19c3733-c7a0-4471-b5f5-95074bf79c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:40.302412 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19c3733-c7a0-4471-b5f5-95074bf79c58 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:40.303153 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19c3733-c7a0-4471-b5f5-95074bf79c58 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:40.303638 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19c3733-c7a0-4471-b5f5-95074bf79c58 None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:40.304587 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19c3733-c7a0-4471-b5f5-95074bf79c58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:40.305076 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19c3733-c7a0-4471-b5f5-95074bf79c58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:40.308135 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19c3733-c7a0-4471-b5f5-95074bf79c58 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:40.330567 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 298/591] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:31:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:40.331852 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19c3733-c7a0-4471-b5f5-95074bf79c58 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:40.363746 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:40.401902 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2daba6-482f-4792-86e9-f0be586daceb None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:40.445011 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca6d982-d6e9-49ef-8013-41f861ee89f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:40.446052 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca6d982-d6e9-49ef-8013-41f861ee89f3 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:40.446334 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca6d982-d6e9-49ef-8013-41f861ee89f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:40.447034 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca6d982-d6e9-49ef-8013-41f861ee89f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:40.448979 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:40.458258 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca6d982-d6e9-49ef-8013-41f861ee89f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:40.459768 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca6d982-d6e9-49ef-8013-41f861ee89f3 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:40.472027 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19c3733-c7a0-4471-b5f5-95074bf79c58 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:40.481551 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-2ca6d982-d6e9-49ef-8013-41f861ee89f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:40.481551 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:40.481551 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:40.481551 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:40.481551 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:40.481551 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:40.481551 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:40.481551 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:40.481551 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:40.481551 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:40.481551 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:40.481551 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:40.481551 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:31:40.481551 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:31:40.481551 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:31:40.482751 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:31:40.482751 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:40.482751 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:40.482751 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:40.482751 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:40.482751 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:40.482751 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:40.482751 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:40.482751 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:40.482751 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:40.482751 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:40.482751 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:40.482751 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:40.482751 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:40.482751 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:40.482751 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:40.482751 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:40.483920 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:31:40.483920 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:31:40.483920 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:40.483920 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:40.483920 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 294/592] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:31:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:40.484691 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca6d982-d6e9-49ef-8013-41f861ee89f3 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:40.510218 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a580da85-731a-458d-b4d3-10f40746407f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:40.510981 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a580da85-731a-458d-b4d3-10f40746407f None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:40.511341 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a580da85-731a-458d-b4d3-10f40746407f None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:40.511723 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a580da85-731a-458d-b4d3-10f40746407f None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:40.512807 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a580da85-731a-458d-b4d3-10f40746407f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:40.513189 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a580da85-731a-458d-b4d3-10f40746407f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:40.514666 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a580da85-731a-458d-b4d3-10f40746407f None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:40.532198 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 299/593] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:31:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:40.532198 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98d4cc1-6492-41b3-b1eb-e1f0c9f6efc1 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:40.532198 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98d4cc1-6492-41b3-b1eb-e1f0c9f6efc1 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:40.532198 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98d4cc1-6492-41b3-b1eb-e1f0c9f6efc1 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:40.532198 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 300/594] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:31:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:40.532198 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98d4cc1-6492-41b3-b1eb-e1f0c9f6efc1 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:40.597853 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca6d982-d6e9-49ef-8013-41f861ee89f3 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:40.623258 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:40.634026 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c38e364-105f-4033-966f-46ddddc96aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:40.634826 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c38e364-105f-4033-966f-46ddddc96aa7 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:40.635106 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c38e364-105f-4033-966f-46ddddc96aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:40.635379 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c38e364-105f-4033-966f-46ddddc96aa7 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:40.643164 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c38e364-105f-4033-966f-46ddddc96aa7 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:40.644636 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c38e364-105f-4033-966f-46ddddc96aa7 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:40.649947 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98d4cc1-6492-41b3-b1eb-e1f0c9f6efc1 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:40.660601 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-8c38e364-105f-4033-966f-46ddddc96aa7 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 15 14:31:40.660601 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:40.660601 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:40.660601 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:40.660601 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:40.660601 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:40.660601 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:40.660601 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:40.660601 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:40.660601 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:40.660601 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:40.660601 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:40.660601 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:31:40.660601 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:31:40.660601 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:31:40.660601 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:31:40.661536 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:40.661536 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:40.661536 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:31:40.661536 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:31:40.661536 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:31:40.661536 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:31:40.661536 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:40.661536 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:40.661536 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:40.661536 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:40.661536 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:40.661536 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:40.661536 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:40.661536 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:40.661536 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:40.661536 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:40.661536 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:40.661536 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:40.662474 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:40.662474 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:40.662474 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:31:40.662474 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:31:40.662474 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:31:40.662474 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:31:40.662474 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:31:40.662474 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:31:40.662474 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 15 14:31:40.662474 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:40.662474 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 295/595] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Jun 15 14:31:40 2022] GET /identity/v3/users/neutron => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:40.662474 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c38e364-105f-4033-966f-46ddddc96aa7 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:40.692642 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb2be02-48f6-4a87-a159-1aec3cfddd7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:40.693307 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb2be02-48f6-4a87-a159-1aec3cfddd7a None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:40.693588 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb2be02-48f6-4a87-a159-1aec3cfddd7a None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:40.693856 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb2be02-48f6-4a87-a159-1aec3cfddd7a None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:40.694579 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb2be02-48f6-4a87-a159-1aec3cfddd7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:40.694894 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb2be02-48f6-4a87-a159-1aec3cfddd7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:40.696279 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb2be02-48f6-4a87-a159-1aec3cfddd7a None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:40.709042 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 301/596] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:31:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:40.710108 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d644403-61cb-48a9-b548-cb26cf5af344 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:40.710396 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d644403-61cb-48a9-b548-cb26cf5af344 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:40.710678 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d644403-61cb-48a9-b548-cb26cf5af344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:40.723612 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:40.727838 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-7d644403-61cb-48a9-b548-cb26cf5af344 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:40.729160 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-7d644403-61cb-48a9-b548-cb26cf5af344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:40.783769 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c38e364-105f-4033-966f-46ddddc96aa7 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:40.802432 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 302/597] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:31:40.803638 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d644403-61cb-48a9-b548-cb26cf5af344 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:40.824477 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0323fd0-54a1-45b7-a7e0-40a0143cc75c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:40.826061 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0323fd0-54a1-45b7-a7e0-40a0143cc75c None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:40.826660 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0323fd0-54a1-45b7-a7e0-40a0143cc75c None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:40.827212 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0323fd0-54a1-45b7-a7e0-40a0143cc75c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:40.834427 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0323fd0-54a1-45b7-a7e0-40a0143cc75c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:40.836062 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0323fd0-54a1-45b7-a7e0-40a0143cc75c None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:40.850708 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-b0323fd0-54a1-45b7-a7e0-40a0143cc75c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:31:40.850708 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:40.850708 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:40.850708 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:40.850708 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:40.850708 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:40.850708 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:40.850708 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:40.850708 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:40.850708 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:40.850708 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:40.850708 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:40.850708 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:31:40.850708 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:31:40.850708 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:31:40.851629 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:31:40.851629 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:40.851629 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:40.851629 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:40.851629 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:40.851629 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:40.851629 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:40.851629 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:40.851629 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:40.851629 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:40.851629 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:40.851629 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:40.851629 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:40.851629 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:40.851629 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:40.851629 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:40.851629 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:40.858410 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:31:40.858410 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:31:40.858410 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:31:40.858410 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:31:40.858410 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:31:40.858410 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:31:40.858410 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:31:40.858410 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:40.858410 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 296/598] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jun 15 14:31:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:40.858410 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0323fd0-54a1-45b7-a7e0-40a0143cc75c None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:40.887158 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:40.924246 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d644403-61cb-48a9-b548-cb26cf5af344 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:40.959088 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0323fd0-54a1-45b7-a7e0-40a0143cc75c None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:40.961073 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed30128-25e3-458c-b88e-14681900ae37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:40.961897 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed30128-25e3-458c-b88e-14681900ae37 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:40.962189 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed30128-25e3-458c-b88e-14681900ae37 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:40.962472 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed30128-25e3-458c-b88e-14681900ae37 None admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:40.963314 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed30128-25e3-458c-b88e-14681900ae37 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:40.963630 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed30128-25e3-458c-b88e-14681900ae37 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:40.965091 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed30128-25e3-458c-b88e-14681900ae37 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:40.967039 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:40.985562 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 303/599] 10.222.0.42 () {56 vars in 1139 bytes} [Wed Jun 15 14:31:40 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:40.986404 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed30128-25e3-458c-b88e-14681900ae37 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:40.999171 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45e0e0b-56e7-4795-990c-43dab88888ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:40.999963 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e0e0b-56e7-4795-990c-43dab88888ec None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:41.000277 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e0e0b-56e7-4795-990c-43dab88888ec None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:41.000553 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e0e0b-56e7-4795-990c-43dab88888ec None admin] PATH_INFO: `/v3/projects/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:41.008107 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45e0e0b-56e7-4795-990c-43dab88888ec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:41.010480 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45e0e0b-56e7-4795-990c-43dab88888ec None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:41.026233 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-c45e0e0b-56e7-4795-990c-43dab88888ec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:31:41.026233 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:41.026233 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:41.026233 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:41.026233 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:41.026233 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:41.026233 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:41.026233 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:41.026233 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:41.026233 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:41.026233 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:41.026233 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:41.026233 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:31:41.026233 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:31:41.026233 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:31:41.027158 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:31:41.027158 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:41.027158 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:41.027158 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:41.027158 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:41.027158 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:41.027158 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:41.027158 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:41.027158 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:41.027158 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:41.027158 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:41.027158 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:41.027158 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:41.027158 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:41.027158 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:41.027158 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:41.027158 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:41.028813 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:31:41.028813 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:31:41.028813 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:31:41.028813 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:31:41.028813 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:31:41.028813 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:31:41.028813 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:31:41.028813 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:41.033463 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 297/600] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jun 15 14:31:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:41.034339 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45e0e0b-56e7-4795-990c-43dab88888ec None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:41.095877 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed30128-25e3-458c-b88e-14681900ae37 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:41.120982 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:41.131295 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f28567-f570-4305-a767-7ac54abb15ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:41.132101 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f28567-f570-4305-a767-7ac54abb15ff None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:41.132415 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f28567-f570-4305-a767-7ac54abb15ff None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:41.132662 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f28567-f570-4305-a767-7ac54abb15ff None admin] PATH_INFO: `/v3/roles/service` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:41.142503 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45e0e0b-56e7-4795-990c-43dab88888ec None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:41.143914 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f28567-f570-4305-a767-7ac54abb15ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:31:41.152315 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-d6f28567-f570-4305-a767-7ac54abb15ff None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:31:41.152315 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:41.152315 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:41.152315 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:41.152315 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:41.152315 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:41.152315 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:41.152315 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:41.152315 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:41.152315 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:41.152315 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:41.152315 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:41.152315 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:31:41.152315 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:31:41.152315 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:31:41.152315 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:31:41.153517 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:31:41.153517 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:31:41.153517 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:31:41.153517 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:31:41.153517 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:41.153517 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:41.153517 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:41.153517 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:41.153517 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:41.153517 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:41.153517 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:41.153517 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:41.153517 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:41.153517 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:41.153517 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:41.153517 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:41.154530 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:41.154530 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:41.154530 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:41.154530 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:41.154530 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:31:41.154530 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:31:41.154530 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:31:41.154530 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:31:41.154530 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:31:41.154530 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:31:41.154530 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:31:41.154530 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:41.154530 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 304/601] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Jun 15 14:31:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:41.155226 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f28567-f570-4305-a767-7ac54abb15ff None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:41.187559 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69e196e-ab8f-42ae-9a1c-d18b5b883eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:41.188191 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e196e-ab8f-42ae-9a1c-d18b5b883eef None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:41.188471 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e196e-ab8f-42ae-9a1c-d18b5b883eef None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:41.188746 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e196e-ab8f-42ae-9a1c-d18b5b883eef None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:41.189527 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69e196e-ab8f-42ae-9a1c-d18b5b883eef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:41.189831 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69e196e-ab8f-42ae-9a1c-d18b5b883eef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:41.191250 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69e196e-ab8f-42ae-9a1c-d18b5b883eef None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:41.207720 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 298/602] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jun 15 14:31:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:41.207720 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69e196e-ab8f-42ae-9a1c-d18b5b883eef None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:41.235132 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:41.264752 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f28567-f570-4305-a767-7ac54abb15ff None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:41.301727 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18a6e8c-41c0-4ae3-8bcb-736932e0ca51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:41.302613 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18a6e8c-41c0-4ae3-8bcb-736932e0ca51 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:41.302904 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18a6e8c-41c0-4ae3-8bcb-736932e0ca51 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:41.303181 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18a6e8c-41c0-4ae3-8bcb-736932e0ca51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:41.312867 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69e196e-ab8f-42ae-9a1c-d18b5b883eef None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:41.314319 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18a6e8c-41c0-4ae3-8bcb-736932e0ca51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:41.315725 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18a6e8c-41c0-4ae3-8bcb-736932e0ca51 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:41.332598 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-f18a6e8c-41c0-4ae3-8bcb-736932e0ca51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:41.332598 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:41.332598 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:41.332598 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:41.332598 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:41.332598 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:41.332598 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:41.332598 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:41.332598 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:41.332598 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:41.332598 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:41.332598 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:41.332598 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:31:41.332598 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:31:41.332598 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:31:41.333530 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:31:41.333530 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:41.333530 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:41.333530 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:41.333530 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:41.333530 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:41.333530 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:41.333530 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:41.333530 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:41.333530 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:41.333530 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:41.333530 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:41.333530 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:41.333530 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:41.333530 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:41.333530 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:41.333530 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:41.334420 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:31:41.334420 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:31:41.334420 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:41.334420 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:41.334420 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 305/603] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:31:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:41.335315 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18a6e8c-41c0-4ae3-8bcb-736932e0ca51 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:41.352461 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad12d0e-86b6-4020-bb89-21b5a06aa07f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:41.353158 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad12d0e-86b6-4020-bb89-21b5a06aa07f None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:41.353441 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad12d0e-86b6-4020-bb89-21b5a06aa07f None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:41.353723 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad12d0e-86b6-4020-bb89-21b5a06aa07f None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:41.354528 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad12d0e-86b6-4020-bb89-21b5a06aa07f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:41.354840 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad12d0e-86b6-4020-bb89-21b5a06aa07f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:41.357041 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad12d0e-86b6-4020-bb89-21b5a06aa07f None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:41.375924 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 299/604] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jun 15 14:31:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:41.375924 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad12d0e-86b6-4020-bb89-21b5a06aa07f None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:41.383013 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:41.455543 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18a6e8c-41c0-4ae3-8bcb-736932e0ca51 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:41.465084 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:41.482043 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad12d0e-86b6-4020-bb89-21b5a06aa07f None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:41.491086 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa9c1a5-fe45-42e5-98d4-d1ce6c8ab9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:41.491746 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa9c1a5-fe45-42e5-98d4-d1ce6c8ab9cd None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:41.492025 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa9c1a5-fe45-42e5-98d4-d1ce6c8ab9cd None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:41.492325 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa9c1a5-fe45-42e5-98d4-d1ce6c8ab9cd None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:41.493065 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa9c1a5-fe45-42e5-98d4-d1ce6c8ab9cd None admin] RBAC: Adding query filter params (name=service) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:41.493366 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa9c1a5-fe45-42e5-98d4-d1ce6c8ab9cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:41.495219 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa9c1a5-fe45-42e5-98d4-d1ce6c8ab9cd None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:41.508176 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 306/605] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Jun 15 14:31:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:41.508176 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa9c1a5-fe45-42e5-98d4-d1ce6c8ab9cd None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:41.524543 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2076cd9-5d3d-4bba-8a03-8f3ba4faddd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:41.525430 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2076cd9-5d3d-4bba-8a03-8f3ba4faddd0 None admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:41.525715 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2076cd9-5d3d-4bba-8a03-8f3ba4faddd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:41.525981 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2076cd9-5d3d-4bba-8a03-8f3ba4faddd0 None admin] PATH_INFO: `/v3/projects/3094ff8966f149bda784da6a51b1358f/users/e44f6cd435be4e03bc79cb3125d71451/roles/305a1ca2e6f740eba7dc696849e933eb` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:41.542048 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2076cd9-5d3d-4bba-8a03-8f3ba4faddd0 None admin] RBAC: Authorizing `identity:create_grant(project_id=3094ff8966f149bda784da6a51b1358f, user_id=e44f6cd435be4e03bc79cb3125d71451, role_id=305a1ca2e6f740eba7dc696849e933eb)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:41.544008 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2076cd9-5d3d-4bba-8a03-8f3ba4faddd0 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:41.611520 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:41.615285 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 300/606] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Jun 15 14:31:41 2022] PUT /identity/v3/projects/3094ff8966f149bda784da6a51b1358f/users/e44f6cd435be4e03bc79cb3125d71451/roles/305a1ca2e6f740eba7dc696849e933eb => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:31:41.616057 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2076cd9-5d3d-4bba-8a03-8f3ba4faddd0 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:41.616233 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa9c1a5-fe45-42e5-98d4-d1ce6c8ab9cd None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:41.662114 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c4544-5fb7-4c28-9a77-e1aaec63d428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:41.663521 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c4544-5fb7-4c28-9a77-e1aaec63d428 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:41.664075 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c4544-5fb7-4c28-9a77-e1aaec63d428 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:41.664522 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c4544-5fb7-4c28-9a77-e1aaec63d428 None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:41.665555 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4c4544-5fb7-4c28-9a77-e1aaec63d428 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:41.666349 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4c4544-5fb7-4c28-9a77-e1aaec63d428 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:41.668304 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4c4544-5fb7-4c28-9a77-e1aaec63d428 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:41.688689 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 307/607] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:31:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:41.690142 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c4544-5fb7-4c28-9a77-e1aaec63d428 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:41.769255 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2076cd9-5d3d-4bba-8a03-8f3ba4faddd0 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:41.818581 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03758010-bedb-445f-ba0f-af86f18571d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:41.819477 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03758010-bedb-445f-ba0f-af86f18571d5 None admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:41.819773 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03758010-bedb-445f-ba0f-af86f18571d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:41.820044 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03758010-bedb-445f-ba0f-af86f18571d5 None admin] PATH_INFO: `/v3/projects/3094ff8966f149bda784da6a51b1358f/users/4f18c57f27784694935bfb0bcea48a78/roles/305a1ca2e6f740eba7dc696849e933eb` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:41.822644 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c4544-5fb7-4c28-9a77-e1aaec63d428 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:41.828278 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:41.847531 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03758010-bedb-445f-ba0f-af86f18571d5 None admin] RBAC: Authorizing `identity:create_grant(project_id=3094ff8966f149bda784da6a51b1358f, user_id=4f18c57f27784694935bfb0bcea48a78, role_id=305a1ca2e6f740eba7dc696849e933eb)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:41.849916 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03758010-bedb-445f-ba0f-af86f18571d5 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:41.865276 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f781a9-e155-45fa-a023-b8b2dfd988db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:41.866223 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f781a9-e155-45fa-a023-b8b2dfd988db None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:41.866617 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f781a9-e155-45fa-a023-b8b2dfd988db None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:41.867042 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f781a9-e155-45fa-a023-b8b2dfd988db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:41.878948 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f781a9-e155-45fa-a023-b8b2dfd988db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:41.880596 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f781a9-e155-45fa-a023-b8b2dfd988db None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:41.899569 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-b7f781a9-e155-45fa-a023-b8b2dfd988db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:41.899569 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:41.899569 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:41.899569 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:41.899569 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:41.899569 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:41.899569 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:41.899569 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:41.899569 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:41.899569 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:41.899569 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:41.899569 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:41.899569 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:31:41.899569 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:31:41.899569 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:31:41.900711 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:31:41.900711 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:41.900711 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:41.900711 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:41.900711 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:41.900711 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:41.900711 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:41.900711 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:41.900711 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:41.900711 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:41.900711 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:41.900711 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:41.900711 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:41.900711 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:41.900711 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:41.900711 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:41.900711 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:41.906880 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:31:41.906880 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:31:41.906880 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:41.906880 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:41.906880 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 308/608] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:31:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:41.906880 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f781a9-e155-45fa-a023-b8b2dfd988db None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:41.922801 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:41.970595 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 301/609] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Jun 15 14:31:41 2022] PUT /identity/v3/projects/3094ff8966f149bda784da6a51b1358f/users/4f18c57f27784694935bfb0bcea48a78/roles/305a1ca2e6f740eba7dc696849e933eb => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:31:41.975774 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03758010-bedb-445f-ba0f-af86f18571d5 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:42.084601 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f781a9-e155-45fa-a023-b8b2dfd988db None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:42.154252 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2918d1d4-a631-4be5-bf16-097ea9db9f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:42.155070 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2918d1d4-a631-4be5-bf16-097ea9db9f86 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:42.155349 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2918d1d4-a631-4be5-bf16-097ea9db9f86 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:42.155629 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2918d1d4-a631-4be5-bf16-097ea9db9f86 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:42.156028 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03758010-bedb-445f-ba0f-af86f18571d5 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:42.167631 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2918d1d4-a631-4be5-bf16-097ea9db9f86 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:42.170015 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2918d1d4-a631-4be5-bf16-097ea9db9f86 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:42.194324 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bffa681-a862-4016-924b-3072665cd151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:42.195040 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bffa681-a862-4016-924b-3072665cd151 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:42.195332 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bffa681-a862-4016-924b-3072665cd151 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:42.195605 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bffa681-a862-4016-924b-3072665cd151 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:42.196362 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bffa681-a862-4016-924b-3072665cd151 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:42.196673 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bffa681-a862-4016-924b-3072665cd151 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:42.198050 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bffa681-a862-4016-924b-3072665cd151 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:42.204511 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-2918d1d4-a631-4be5-bf16-097ea9db9f86 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:31:42.204511 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:42.204511 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:42.204511 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:42.204511 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:42.204511 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:42.204511 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:42.204511 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:42.204511 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:42.204511 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:42.204511 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:42.204511 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:42.204511 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:31:42.204511 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:31:42.204511 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:31:42.205499 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:31:42.205499 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:42.205499 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:42.205499 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:42.205499 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:42.205499 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:42.205499 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:42.205499 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:42.205499 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:42.205499 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:42.205499 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:42.205499 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:42.205499 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:42.205499 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:42.205499 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:42.205499 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:42.205499 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:42.206429 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:31:42.206429 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:31:42.206429 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:31:42.206429 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:31:42.206429 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:31:42.206429 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:31:42.206429 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:31:42.206429 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:42.206429 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 309/610] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jun 15 14:31:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:42.211179 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2918d1d4-a631-4be5-bf16-097ea9db9f86 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:42.217438 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 302/611] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:31:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:42.223128 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bffa681-a862-4016-924b-3072665cd151 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:42.317376 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:42.344639 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:42.345318 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bffa681-a862-4016-924b-3072665cd151 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:42.384426 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb0b6ee-d251-480a-b6e7-4c4c05b14f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:42.385077 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb0b6ee-d251-480a-b6e7-4c4c05b14f3e None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:42.385357 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb0b6ee-d251-480a-b6e7-4c4c05b14f3e None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:42.385652 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb0b6ee-d251-480a-b6e7-4c4c05b14f3e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:42.396368 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb0b6ee-d251-480a-b6e7-4c4c05b14f3e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:42.398209 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb0b6ee-d251-480a-b6e7-4c4c05b14f3e None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:42.416416 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-ccb0b6ee-d251-480a-b6e7-4c4c05b14f3e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 15 14:31:42.416416 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:42.416416 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:42.416416 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:42.416416 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:42.416416 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:42.416416 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:42.416416 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:42.416416 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:42.416416 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:42.416416 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:42.416416 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:42.416416 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:31:42.416416 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:31:42.416416 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:31:42.416416 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:31:42.417432 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:42.417432 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:42.417432 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:31:42.417432 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:31:42.417432 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:31:42.417432 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:31:42.417432 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:42.417432 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:42.417432 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:42.417432 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:42.417432 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:42.417432 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:42.417432 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:42.417432 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:42.417432 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:42.417432 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:42.417432 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:42.417432 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:42.419002 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:42.419002 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:42.419002 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:31:42.419002 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:31:42.419002 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:31:42.419002 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:31:42.419002 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:31:42.419002 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:31:42.419002 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 15 14:31:42.419002 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:42.419002 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 303/612] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jun 15 14:31:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:31:42.426551 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb0b6ee-d251-480a-b6e7-4c4c05b14f3e None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:42.432913 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2918d1d4-a631-4be5-bf16-097ea9db9f86 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:42.486792 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030af97a-3176-48b5-b361-31fff63f2daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:42.487567 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030af97a-3176-48b5-b361-31fff63f2daa None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:42.487941 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030af97a-3176-48b5-b361-31fff63f2daa None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:42.488339 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030af97a-3176-48b5-b361-31fff63f2daa None admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:42.489211 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030af97a-3176-48b5-b361-31fff63f2daa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:42.489593 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030af97a-3176-48b5-b361-31fff63f2daa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:42.491104 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030af97a-3176-48b5-b361-31fff63f2daa None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:42.505925 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 310/613] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jun 15 14:31:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:31:42.511892 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c48d2-b5b1-445e-be1e-1386b7e060bd None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:42.511892 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c48d2-b5b1-445e-be1e-1386b7e060bd None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:42.511892 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c48d2-b5b1-445e-be1e-1386b7e060bd None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:42.514300 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 311/614] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:31:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:42.515131 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861c48d2-b5b1-445e-be1e-1386b7e060bd None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:42.578999 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb0b6ee-d251-480a-b6e7-4c4c05b14f3e None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:42.588820 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:42.627538 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cd54ff-4237-4a46-8bf0-d683fd8eed98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:42.628793 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cd54ff-4237-4a46-8bf0-d683fd8eed98 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:42.629161 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cd54ff-4237-4a46-8bf0-d683fd8eed98 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:42.629539 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cd54ff-4237-4a46-8bf0-d683fd8eed98 None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:42.630534 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cd54ff-4237-4a46-8bf0-d683fd8eed98 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:42.630945 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cd54ff-4237-4a46-8bf0-d683fd8eed98 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:42.632800 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cd54ff-4237-4a46-8bf0-d683fd8eed98 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:42.639103 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861c48d2-b5b1-445e-be1e-1386b7e060bd None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:42.656002 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 304/615] 10.222.0.42 () {56 vars in 1135 bytes} [Wed Jun 15 14:31:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:42.657460 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b224db-ecb3-49c6-a3dc-8fcafe669ccc None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:42.657751 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b224db-ecb3-49c6-a3dc-8fcafe669ccc None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:42.658029 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b224db-ecb3-49c6-a3dc-8fcafe669ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:42.670398 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:42.676187 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62baff8-1985-4e8a-8041-418bbd455907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:42.677105 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62baff8-1985-4e8a-8041-418bbd455907 None admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:42.677296 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-d0b224db-ecb3-49c6-a3dc-8fcafe669ccc None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:42.677429 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62baff8-1985-4e8a-8041-418bbd455907 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:42.677649 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62baff8-1985-4e8a-8041-418bbd455907 None admin] PATH_INFO: `/v3/projects/3094ff8966f149bda784da6a51b1358f/users/c739b61c8a8b453e979bda720bafebb1/roles/305a1ca2e6f740eba7dc696849e933eb` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:42.678983 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-d0b224db-ecb3-49c6-a3dc-8fcafe669ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:42.693687 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62baff8-1985-4e8a-8041-418bbd455907 None admin] RBAC: Authorizing `identity:create_grant(project_id=3094ff8966f149bda784da6a51b1358f, user_id=c739b61c8a8b453e979bda720bafebb1, role_id=305a1ca2e6f740eba7dc696849e933eb)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:42.695399 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62baff8-1985-4e8a-8041-418bbd455907 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:42.752894 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 305/616] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:31:42.754039 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b224db-ecb3-49c6-a3dc-8fcafe669ccc None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:42.781174 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 312/617] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Jun 15 14:31:42 2022] PUT /identity/v3/projects/3094ff8966f149bda784da6a51b1358f/users/c739b61c8a8b453e979bda720bafebb1/roles/305a1ca2e6f740eba7dc696849e933eb => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:31:42.799307 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62baff8-1985-4e8a-8041-418bbd455907 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:42.871047 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:42.879999 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b224db-ecb3-49c6-a3dc-8fcafe669ccc None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:42.920304 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afb142b-28bf-4a96-9d92-38367af7d21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:42.921547 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb142b-28bf-4a96-9d92-38367af7d21d None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:42.922019 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb142b-28bf-4a96-9d92-38367af7d21d None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:42.922479 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb142b-28bf-4a96-9d92-38367af7d21d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:42.935513 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afb142b-28bf-4a96-9d92-38367af7d21d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:42.940361 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afb142b-28bf-4a96-9d92-38367af7d21d None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:42.957888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-0afb142b-28bf-4a96-9d92-38367af7d21d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:42.957888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:42.957888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:42.957888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:42.957888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:42.957888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:42.957888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:42.957888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:42.957888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:42.957888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:42.957888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:42.957888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:42.957888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:31:42.957888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:31:42.957888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:31:42.958848 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:31:42.958848 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:42.958848 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:42.958848 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:42.958848 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:42.958848 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:42.958848 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:42.958848 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:42.958848 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:42.958848 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:42.958848 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:42.958848 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:42.958848 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:42.958848 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:42.958848 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:42.958848 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:42.958848 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:42.959892 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:31:42.959892 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:31:42.959892 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:42.959892 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:42.962086 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 306/618] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:31:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:42.969055 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afb142b-28bf-4a96-9d92-38367af7d21d None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:42.999824 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62baff8-1985-4e8a-8041-418bbd455907 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:43.077959 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2388f5e0-40c6-4b27-ad9f-dc2837c784e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:43.078779 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2388f5e0-40c6-4b27-ad9f-dc2837c784e7 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:43.079060 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2388f5e0-40c6-4b27-ad9f-dc2837c784e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:43.079334 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2388f5e0-40c6-4b27-ad9f-dc2837c784e7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:43.106902 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2388f5e0-40c6-4b27-ad9f-dc2837c784e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:31:43.114938 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-2388f5e0-40c6-4b27-ad9f-dc2837c784e7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:31:43.114938 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:43.114938 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:43.114938 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:43.114938 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:43.114938 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:43.114938 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:43.114938 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:43.114938 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:43.114938 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:43.114938 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:43.114938 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:43.114938 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:31:43.114938 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:31:43.114938 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:31:43.114938 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:31:43.115876 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:31:43.115876 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:31:43.115876 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:31:43.115876 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:31:43.115876 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:43.115876 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:43.115876 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:43.115876 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:43.115876 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:43.115876 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:43.115876 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:43.115876 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:43.115876 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:43.115876 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:43.115876 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:43.115876 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:43.117124 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:43.117124 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:43.117124 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:43.117124 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:43.117124 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:31:43.117124 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:31:43.117124 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:31:43.117124 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:31:43.117124 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:31:43.117124 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:31:43.117124 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:31:43.117124 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:43.117124 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 313/619] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Jun 15 14:31:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:43.124828 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2388f5e0-40c6-4b27-ad9f-dc2837c784e7 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:43.142556 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afb142b-28bf-4a96-9d92-38367af7d21d None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:43.174037 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:43.188440 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8e37ac-a694-4765-9c2f-aaddf8cfab09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:43.189136 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e37ac-a694-4765-9c2f-aaddf8cfab09 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:43.189409 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e37ac-a694-4765-9c2f-aaddf8cfab09 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:43.189704 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e37ac-a694-4765-9c2f-aaddf8cfab09 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:43.190406 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8e37ac-a694-4765-9c2f-aaddf8cfab09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:43.190710 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8e37ac-a694-4765-9c2f-aaddf8cfab09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:43.192132 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8e37ac-a694-4765-9c2f-aaddf8cfab09 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:43.210026 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 307/620] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:31:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:31:43.220208 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8e37ac-a694-4765-9c2f-aaddf8cfab09 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:43.232833 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:43.270574 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2388f5e0-40c6-4b27-ad9f-dc2837c784e7 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:43.336843 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77e1b73-aeb3-4f67-ade1-15dedbfe7656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:43.337589 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77e1b73-aeb3-4f67-ade1-15dedbfe7656 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:43.337873 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77e1b73-aeb3-4f67-ade1-15dedbfe7656 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:43.338149 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77e1b73-aeb3-4f67-ade1-15dedbfe7656 None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:43.338906 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77e1b73-aeb3-4f67-ade1-15dedbfe7656 None admin] RBAC: Adding query filter params (name=service) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:43.339219 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77e1b73-aeb3-4f67-ade1-15dedbfe7656 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:43.340666 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77e1b73-aeb3-4f67-ade1-15dedbfe7656 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:43.349521 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 314/621] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Jun 15 14:31:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:31:43.358498 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77e1b73-aeb3-4f67-ade1-15dedbfe7656 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:43.393761 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8e37ac-a694-4765-9c2f-aaddf8cfab09 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:43.430193 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c903b498-e527-4a7f-82df-bcb3df2b005a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:43.431422 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903b498-e527-4a7f-82df-bcb3df2b005a None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:43.431951 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903b498-e527-4a7f-82df-bcb3df2b005a None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:43.432461 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903b498-e527-4a7f-82df-bcb3df2b005a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:43.440985 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c903b498-e527-4a7f-82df-bcb3df2b005a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:43.442735 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c903b498-e527-4a7f-82df-bcb3df2b005a None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:43.458391 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-c903b498-e527-4a7f-82df-bcb3df2b005a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:31:43.458391 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:43.458391 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:43.458391 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:43.458391 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:43.458391 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:43.458391 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:43.458391 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:43.458391 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:43.458391 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:43.458391 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:43.458391 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:43.458391 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:31:43.458391 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:31:43.458391 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:31:43.459365 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:31:43.459365 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:43.459365 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:43.459365 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:43.459365 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:43.459365 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:43.459365 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:43.459365 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:43.459365 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:43.459365 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:43.459365 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:43.459365 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:43.459365 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:43.459365 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:43.459365 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:43.459365 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:43.459365 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:43.460256 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:31:43.460256 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:31:43.460256 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:31:43.460256 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:31:43.460256 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:31:43.460256 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:31:43.460256 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:31:43.460256 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:43.466434 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 308/622] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jun 15 14:31:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:43.469005 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c903b498-e527-4a7f-82df-bcb3df2b005a None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:43.528146 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77e1b73-aeb3-4f67-ade1-15dedbfe7656 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:43.540099 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:43.542827 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:43.579886 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c903b498-e527-4a7f-82df-bcb3df2b005a None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:43.591997 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3530e3e7-9237-42ad-9a02-50e8897fe358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:43.591997 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3530e3e7-9237-42ad-9a02-50e8897fe358 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:43.591997 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3530e3e7-9237-42ad-9a02-50e8897fe358 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:43.591997 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3530e3e7-9237-42ad-9a02-50e8897fe358 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:43.613397 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3530e3e7-9237-42ad-9a02-50e8897fe358 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:43.615188 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3530e3e7-9237-42ad-9a02-50e8897fe358 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:43.625368 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f43817-266e-4a4b-9f7a-d7488f14a8cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:43.626387 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f43817-266e-4a4b-9f7a-d7488f14a8cc None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:43.626925 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f43817-266e-4a4b-9f7a-d7488f14a8cc None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:43.627406 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f43817-266e-4a4b-9f7a-d7488f14a8cc None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:43.628417 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f43817-266e-4a4b-9f7a-d7488f14a8cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:43.628942 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f43817-266e-4a4b-9f7a-d7488f14a8cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:43.630634 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f43817-266e-4a4b-9f7a-d7488f14a8cc None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:43.635161 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-3530e3e7-9237-42ad-9a02-50e8897fe358 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:43.635161 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:43.635161 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:43.635161 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:43.635161 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:43.635161 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:43.635161 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:43.635161 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:43.635161 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:43.635161 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:43.635161 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:43.635161 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:43.635161 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:31:43.635161 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:31:43.635161 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:31:43.639747 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:31:43.639747 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:43.639747 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:43.639747 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:43.639747 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:43.639747 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:43.639747 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:43.639747 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:43.639747 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:43.639747 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:43.639747 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:43.639747 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:43.639747 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:43.639747 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:43.639747 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:43.639747 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:43.639747 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:43.640739 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:31:43.640739 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:31:43.640739 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:43.640739 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:43.640739 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 315/623] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:31:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:43.640739 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3530e3e7-9237-42ad-9a02-50e8897fe358 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:43.664454 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 309/624] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jun 15 14:31:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:31:43.669799 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f43817-266e-4a4b-9f7a-d7488f14a8cc None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:43.744451 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3530e3e7-9237-42ad-9a02-50e8897fe358 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:43.780344 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab361d3b-ca7f-497c-b99b-683df1f817a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:43.781164 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab361d3b-ca7f-497c-b99b-683df1f817a0 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:43.781421 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab361d3b-ca7f-497c-b99b-683df1f817a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:43.781583 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab361d3b-ca7f-497c-b99b-683df1f817a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:43.782284 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab361d3b-ca7f-497c-b99b-683df1f817a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:43.782595 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab361d3b-ca7f-497c-b99b-683df1f817a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:43.783975 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab361d3b-ca7f-497c-b99b-683df1f817a0 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:43.796780 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 316/625] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:31:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:43.802088 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab361d3b-ca7f-497c-b99b-683df1f817a0 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:43.828509 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:43.904035 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f43817-266e-4a4b-9f7a-d7488f14a8cc None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:43.962197 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:43.967864 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab361d3b-ca7f-497c-b99b-683df1f817a0 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:43.974049 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e907c829-5eec-4cc6-8eb6-2b1512b5c282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:43.974969 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e907c829-5eec-4cc6-8eb6-2b1512b5c282 None admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:43.975255 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e907c829-5eec-4cc6-8eb6-2b1512b5c282 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:43.975536 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e907c829-5eec-4cc6-8eb6-2b1512b5c282 None admin] PATH_INFO: `/v3/projects/3094ff8966f149bda784da6a51b1358f/users/c04dc5832333498e96a7e02bbfb9484c/roles/305a1ca2e6f740eba7dc696849e933eb` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:43.993520 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e907c829-5eec-4cc6-8eb6-2b1512b5c282 None admin] RBAC: Authorizing `identity:create_grant(project_id=3094ff8966f149bda784da6a51b1358f, user_id=c04dc5832333498e96a7e02bbfb9484c, role_id=305a1ca2e6f740eba7dc696849e933eb)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:43.995071 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e907c829-5eec-4cc6-8eb6-2b1512b5c282 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:44.008165 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2a8164-88f0-4c0a-997f-82115a59cf1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:44.008959 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2a8164-88f0-4c0a-997f-82115a59cf1d None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:44.009416 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2a8164-88f0-4c0a-997f-82115a59cf1d None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:44.009705 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2a8164-88f0-4c0a-997f-82115a59cf1d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:44.017910 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2a8164-88f0-4c0a-997f-82115a59cf1d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:44.019331 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2a8164-88f0-4c0a-997f-82115a59cf1d None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:44.034021 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-fd2a8164-88f0-4c0a-997f-82115a59cf1d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:31:44.034021 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:44.034021 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:44.034021 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:44.034021 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:44.034021 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:44.034021 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:44.034021 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:44.034021 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:44.034021 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:44.034021 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:44.034021 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:44.034021 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:31:44.034021 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:31:44.034021 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:31:44.034021 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:31:44.034990 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:44.034990 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:44.034990 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:31:44.034990 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:31:44.034990 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:31:44.034990 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:31:44.034990 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:44.034990 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:44.034990 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:44.034990 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:44.034990 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:44.034990 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:44.034990 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:44.034990 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:44.034990 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:44.034990 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:44.034990 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:44.034990 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:44.036592 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:44.036592 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:44.036592 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:31:44.036592 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:31:44.036592 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:31:44.036592 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:31:44.036592 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:31:44.036592 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:31:44.036592 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:31:44.036592 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:44.036592 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 317/626] 10.222.0.42 () {56 vars in 1093 bytes} [Wed Jun 15 14:31:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:31:44.036592 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae28fe-9523-43ec-8d5c-b020d532ceaa None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:44.037386 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae28fe-9523-43ec-8d5c-b020d532ceaa None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:44.037386 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae28fe-9523-43ec-8d5c-b020d532ceaa None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:44.038451 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 318/627] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:31:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:44.043170 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afae28fe-9523-43ec-8d5c-b020d532ceaa None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:44.065686 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 310/628] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Jun 15 14:31:43 2022] PUT /identity/v3/projects/3094ff8966f149bda784da6a51b1358f/users/c04dc5832333498e96a7e02bbfb9484c/roles/305a1ca2e6f740eba7dc696849e933eb => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:31:44.067732 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f568ed19-8872-4ff5-9383-7877a78c3029 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:44.068076 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f568ed19-8872-4ff5-9383-7877a78c3029 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:44.068412 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f568ed19-8872-4ff5-9383-7877a78c3029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:44.111184 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-f568ed19-8872-4ff5-9383-7877a78c3029 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:44.114164 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-f568ed19-8872-4ff5-9383-7877a78c3029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:44.192054 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:44.241429 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afae28fe-9523-43ec-8d5c-b020d532ceaa None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:44.264643 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 311/629] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:31:44.288885 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d00690-b5e9-46e0-9c2d-a73e62447b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:44.289563 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d00690-b5e9-46e0-9c2d-a73e62447b7a None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:44.289845 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d00690-b5e9-46e0-9c2d-a73e62447b7a None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:44.290122 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d00690-b5e9-46e0-9c2d-a73e62447b7a None admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:44.290989 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d00690-b5e9-46e0-9c2d-a73e62447b7a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:44.291300 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d00690-b5e9-46e0-9c2d-a73e62447b7a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:44.292753 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d00690-b5e9-46e0-9c2d-a73e62447b7a None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:44.294127 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f568ed19-8872-4ff5-9383-7877a78c3029 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:44.320118 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 319/630] 10.222.0.42 () {56 vars in 1133 bytes} [Wed Jun 15 14:31:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:44.322071 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d00690-b5e9-46e0-9c2d-a73e62447b7a None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:44.438896 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f568ed19-8872-4ff5-9383-7877a78c3029 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:44.450409 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:44.519602 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6324e21-61af-4fab-bd1b-10599dc0199b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:44.520448 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6324e21-61af-4fab-bd1b-10599dc0199b None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:44.520735 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6324e21-61af-4fab-bd1b-10599dc0199b None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:44.521121 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6324e21-61af-4fab-bd1b-10599dc0199b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:44.532398 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d00690-b5e9-46e0-9c2d-a73e62447b7a None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:44.532979 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6324e21-61af-4fab-bd1b-10599dc0199b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:31:44.541134 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-d6324e21-61af-4fab-bd1b-10599dc0199b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:31:44.541134 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:44.541134 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:44.541134 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:44.541134 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:44.541134 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:44.541134 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:44.541134 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:44.541134 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:44.541134 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:44.541134 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:44.541134 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:44.541134 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:31:44.541134 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:31:44.541134 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:31:44.541134 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:31:44.542101 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:31:44.542101 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:31:44.542101 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:31:44.542101 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:31:44.542101 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:44.542101 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:44.542101 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:44.542101 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:44.542101 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:44.542101 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:44.542101 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:44.542101 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:44.542101 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:44.542101 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:44.542101 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:44.542101 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:44.542927 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:44.542927 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:44.542927 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:44.542927 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:44.542927 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:31:44.542927 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:31:44.542927 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:31:44.542927 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:31:44.542927 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:31:44.542927 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:31:44.542927 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:31:44.542927 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:44.542927 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 312/631] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Jun 15 14:31:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:31:44.544824 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e1d03-9039-4078-82d3-cc1c69f6d21a None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:44.544920 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e1d03-9039-4078-82d3-cc1c69f6d21a None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:44.545169 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e1d03-9039-4078-82d3-cc1c69f6d21a None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:44.556112 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 313/632] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:31:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:44.556112 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783e1d03-9039-4078-82d3-cc1c69f6d21a None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:44.576842 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:44.581229 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92346ab-a855-4804-8b13-258f495acf8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:44.581878 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92346ab-a855-4804-8b13-258f495acf8d None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:44.582315 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92346ab-a855-4804-8b13-258f495acf8d None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:44.582595 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92346ab-a855-4804-8b13-258f495acf8d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:44.673085 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92346ab-a855-4804-8b13-258f495acf8d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:44.673651 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783e1d03-9039-4078-82d3-cc1c69f6d21a None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:44.675167 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92346ab-a855-4804-8b13-258f495acf8d None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:44.713192 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e47f66-e087-4309-90ab-dd12bd2c059b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:44.714232 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e47f66-e087-4309-90ab-dd12bd2c059b None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:44.714750 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e47f66-e087-4309-90ab-dd12bd2c059b None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:44.716337 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e47f66-e087-4309-90ab-dd12bd2c059b None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:44.717357 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e47f66-e087-4309-90ab-dd12bd2c059b None admin] RBAC: Adding query filter params (name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:44.718316 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e47f66-e087-4309-90ab-dd12bd2c059b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:44.720347 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e47f66-e087-4309-90ab-dd12bd2c059b None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:44.732902 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 314/633] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Jun 15 14:31:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:44.735167 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc53c8b-2d9f-40ee-b9ff-beeca3bc0fd1 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:44.735655 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc53c8b-2d9f-40ee-b9ff-beeca3bc0fd1 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:44.736347 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc53c8b-2d9f-40ee-b9ff-beeca3bc0fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:44.758387 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-7dc53c8b-2d9f-40ee-b9ff-beeca3bc0fd1 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:44.760154 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-7dc53c8b-2d9f-40ee-b9ff-beeca3bc0fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:44.785947 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 320/634] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jun 15 14:31:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:31:44.787158 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92346ab-a855-4804-8b13-258f495acf8d None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:44.792530 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:44.825528 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:44.844549 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 315/635] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:31:44.846477 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc53c8b-2d9f-40ee-b9ff-beeca3bc0fd1 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:44.915124 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92346ab-a855-4804-8b13-258f495acf8d None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:44.952803 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2372348-80ed-4f87-b965-426e17e54501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:44.953729 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2372348-80ed-4f87-b965-426e17e54501 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:44.954112 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2372348-80ed-4f87-b965-426e17e54501 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:44.954464 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2372348-80ed-4f87-b965-426e17e54501 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:44.966544 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2372348-80ed-4f87-b965-426e17e54501 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:44.968208 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2372348-80ed-4f87-b965-426e17e54501 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:44.987975 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-c2372348-80ed-4f87-b965-426e17e54501 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:44.987975 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:44.987975 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:44.987975 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:44.987975 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:44.987975 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:44.987975 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:44.987975 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:44.987975 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:44.987975 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:44.987975 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:44.987975 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:44.987975 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:31:44.987975 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:31:44.987975 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:31:44.988951 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:31:44.988951 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:44.988951 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:44.988951 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:44.988951 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:44.988951 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:44.988951 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:44.988951 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:44.988951 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:44.988951 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:44.988951 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:44.988951 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:44.988951 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:44.988951 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:44.988951 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:44.988951 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:44.988951 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:44.989856 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:31:44.989856 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:31:44.989856 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:44.989856 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:44.989856 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 321/636] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:31:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:44.991029 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2372348-80ed-4f87-b965-426e17e54501 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:44.994890 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc53c8b-2d9f-40ee-b9ff-beeca3bc0fd1 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:45.033825 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d046f84-cf06-47d1-9207-2f4834fb1349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:45.034712 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d046f84-cf06-47d1-9207-2f4834fb1349 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:45.035108 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d046f84-cf06-47d1-9207-2f4834fb1349 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:45.035454 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d046f84-cf06-47d1-9207-2f4834fb1349 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:45.046843 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d046f84-cf06-47d1-9207-2f4834fb1349 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:45.048527 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d046f84-cf06-47d1-9207-2f4834fb1349 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:45.068049 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-3d046f84-cf06-47d1-9207-2f4834fb1349 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:45.068049 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:45.068049 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:45.068049 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:45.068049 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:45.068049 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:45.068049 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:45.068049 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:45.068049 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:45.068049 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:45.068049 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:45.068049 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:45.068049 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:31:45.068049 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:31:45.068049 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:31:45.069019 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:31:45.069019 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:45.069019 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:45.069019 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:45.069019 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:45.069019 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:45.069019 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:45.069019 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:45.069019 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:45.069019 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:45.069019 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:45.069019 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:45.069019 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:45.069019 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:45.069019 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:45.069019 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:45.069019 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:45.069901 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:31:45.069901 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:31:45.069901 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:45.069901 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:45.074224 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 316/637] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:31:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:45.075177 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d046f84-cf06-47d1-9207-2f4834fb1349 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:45.099124 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:45.099807 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:45.120666 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2372348-80ed-4f87-b965-426e17e54501 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:45.171858 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433c61ed-9529-43f2-b390-d1de397ad3a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:45.173009 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433c61ed-9529-43f2-b390-d1de397ad3a7 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:45.173466 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433c61ed-9529-43f2-b390-d1de397ad3a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:45.174441 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433c61ed-9529-43f2-b390-d1de397ad3a7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:45.186683 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d046f84-cf06-47d1-9207-2f4834fb1349 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:45.188465 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433c61ed-9529-43f2-b390-d1de397ad3a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:31:45.198992 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-433c61ed-9529-43f2-b390-d1de397ad3a7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:31:45.198992 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:45.198992 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:45.198992 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:45.198992 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:45.198992 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:45.198992 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:45.198992 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:45.198992 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:45.198992 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:45.198992 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:45.198992 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:45.198992 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:31:45.198992 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:31:45.198992 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:31:45.198992 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:31:45.200117 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:31:45.200117 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:31:45.200117 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:31:45.200117 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:31:45.200117 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:45.200117 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:45.200117 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:45.200117 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:45.200117 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:45.200117 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:45.200117 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:45.200117 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:45.200117 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:45.200117 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:45.200117 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:45.200117 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:45.201068 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:45.201068 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:45.201068 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:45.201068 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:45.201068 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:31:45.201068 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:31:45.201068 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:31:45.201068 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:31:45.201068 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:31:45.201068 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:31:45.201068 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:31:45.201068 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:45.203991 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 322/638] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Jun 15 14:31:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:45.204945 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433c61ed-9529-43f2-b390-d1de397ad3a7 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:45.232595 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088acc61-316e-4ffa-bc2e-8daa43faa0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:45.233501 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088acc61-316e-4ffa-bc2e-8daa43faa0c7 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:45.233935 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088acc61-316e-4ffa-bc2e-8daa43faa0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:45.234380 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088acc61-316e-4ffa-bc2e-8daa43faa0c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:45.235262 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088acc61-316e-4ffa-bc2e-8daa43faa0c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:45.235717 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088acc61-316e-4ffa-bc2e-8daa43faa0c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:45.237246 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088acc61-316e-4ffa-bc2e-8daa43faa0c7 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:45.249720 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 317/639] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:31:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:45.250718 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088acc61-316e-4ffa-bc2e-8daa43faa0c7 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:45.316155 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433c61ed-9529-43f2-b390-d1de397ad3a7 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:45.325717 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:45.353066 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a23ebe3-3878-435e-be21-cbc11e9ee8f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:45.353790 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a23ebe3-3878-435e-be21-cbc11e9ee8f4 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:45.354159 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a23ebe3-3878-435e-be21-cbc11e9ee8f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:45.354532 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a23ebe3-3878-435e-be21-cbc11e9ee8f4 None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:45.355351 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a23ebe3-3878-435e-be21-cbc11e9ee8f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:45.356551 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a23ebe3-3878-435e-be21-cbc11e9ee8f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:45.358119 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a23ebe3-3878-435e-be21-cbc11e9ee8f4 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:45.360907 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088acc61-316e-4ffa-bc2e-8daa43faa0c7 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:45.375987 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 323/640] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:31:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:45.377734 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a23ebe3-3878-435e-be21-cbc11e9ee8f4 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:45.382185 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:45.397115 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc55ac3-c390-4f0c-80f8-3995d3fe204c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:45.398060 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc55ac3-c390-4f0c-80f8-3995d3fe204c None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:45.398552 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc55ac3-c390-4f0c-80f8-3995d3fe204c None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:45.399296 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc55ac3-c390-4f0c-80f8-3995d3fe204c None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:45.400274 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc55ac3-c390-4f0c-80f8-3995d3fe204c None admin] RBAC: Adding query filter params (name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:45.400798 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc55ac3-c390-4f0c-80f8-3995d3fe204c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:45.402378 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc55ac3-c390-4f0c-80f8-3995d3fe204c None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:45.411586 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 318/641] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Jun 15 14:31:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:45.412735 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc55ac3-c390-4f0c-80f8-3995d3fe204c None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:45.502642 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a23ebe3-3878-435e-be21-cbc11e9ee8f4 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:45.523394 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc55ac3-c390-4f0c-80f8-3995d3fe204c None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:45.538840 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ce6fad-29f9-485b-99cc-5279908b297a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:45.539642 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ce6fad-29f9-485b-99cc-5279908b297a None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:45.539928 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ce6fad-29f9-485b-99cc-5279908b297a None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:45.540237 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ce6fad-29f9-485b-99cc-5279908b297a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:45.548899 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ce6fad-29f9-485b-99cc-5279908b297a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:45.550348 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ce6fad-29f9-485b-99cc-5279908b297a None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:45.562492 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:45.565929 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-34ce6fad-29f9-485b-99cc-5279908b297a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 14:31:45.565929 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:45.565929 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:45.565929 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:45.565929 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:45.565929 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:45.565929 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:45.565929 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:45.565929 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:45.565929 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:45.565929 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:45.565929 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:45.565929 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:31:45.565929 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:31:45.565929 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:31:45.565929 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:31:45.567008 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:45.567008 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:45.567008 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:31:45.567008 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:31:45.567008 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:31:45.567008 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:31:45.567008 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:45.567008 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:45.567008 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:45.567008 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:45.567008 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:45.567008 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:45.567008 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:45.567008 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:45.567008 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:45.567008 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:45.567008 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:45.567008 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:45.567997 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:45.567997 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:45.567997 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:31:45.567997 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:31:45.567997 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:31:45.567997 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:31:45.567997 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:31:45.567997 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:31:45.567997 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 14:31:45.567997 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:45.567997 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 324/642] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jun 15 14:31:45 2022] GET /identity/v3/users/cinder => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:45.567997 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ce6fad-29f9-485b-99cc-5279908b297a None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:45.570620 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089584e9-f9a4-4162-a0b3-d54345e2a630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:45.571411 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089584e9-f9a4-4162-a0b3-d54345e2a630 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:45.571700 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089584e9-f9a4-4162-a0b3-d54345e2a630 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:45.571977 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089584e9-f9a4-4162-a0b3-d54345e2a630 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:45.579011 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089584e9-f9a4-4162-a0b3-d54345e2a630 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:45.580499 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089584e9-f9a4-4162-a0b3-d54345e2a630 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:45.595542 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-089584e9-f9a4-4162-a0b3-d54345e2a630 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:31:45.595542 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:45.595542 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:45.595542 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:45.595542 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:45.595542 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:45.595542 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:45.595542 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:45.595542 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:45.595542 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:45.595542 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:45.595542 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:45.595542 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:31:45.595542 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:31:45.595542 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:31:45.596654 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:31:45.596654 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:45.596654 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:45.596654 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:45.596654 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:45.596654 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:45.596654 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:45.596654 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:45.596654 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:45.596654 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:45.596654 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:45.596654 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:45.596654 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:45.596654 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:45.596654 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:45.596654 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:45.596654 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:45.597888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:31:45.597888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:31:45.597888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:31:45.597888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:31:45.597888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:31:45.597888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:31:45.597888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:31:45.597888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:45.597888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 319/643] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jun 15 14:31:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:45.598376 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda0843b-882f-4f6b-963a-6498ab4b4021 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:45.598532 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda0843b-882f-4f6b-963a-6498ab4b4021 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:45.598810 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda0843b-882f-4f6b-963a-6498ab4b4021 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:45.600294 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 320/644] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:31:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:45.604808 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda0843b-882f-4f6b-963a-6498ab4b4021 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:45.645074 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:45.684601 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ce6fad-29f9-485b-99cc-5279908b297a None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:45.717562 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda0843b-882f-4f6b-963a-6498ab4b4021 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:45.720259 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae8bbec-4b80-42af-a454-7c3ba44d29b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:45.721151 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae8bbec-4b80-42af-a454-7c3ba44d29b4 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:45.721440 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae8bbec-4b80-42af-a454-7c3ba44d29b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:45.721716 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae8bbec-4b80-42af-a454-7c3ba44d29b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:45.733257 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae8bbec-4b80-42af-a454-7c3ba44d29b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:45.734699 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae8bbec-4b80-42af-a454-7c3ba44d29b4 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:45.753384 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d503181-d3be-465e-ad91-0b4c46938047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:45.753576 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-2ae8bbec-4b80-42af-a454-7c3ba44d29b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:45.753576 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:45.753576 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:45.753576 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:45.753576 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:45.753576 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:45.753576 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:45.753576 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:45.753576 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:45.753576 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:45.753576 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:45.753576 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:45.753576 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:31:45.753576 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:31:45.753576 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:31:45.754419 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:31:45.754419 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:45.754419 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:45.754419 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:45.754419 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:45.754419 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:45.754419 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:45.754419 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:45.754419 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:45.754419 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:45.754419 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:45.754419 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:45.754419 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:45.754419 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:45.754419 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:45.754419 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:45.754419 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:45.760320 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:31:45.760320 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:31:45.760320 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:45.760320 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:45.760600 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d503181-d3be-465e-ad91-0b4c46938047 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:45.760600 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d503181-d3be-465e-ad91-0b4c46938047 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:45.760600 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d503181-d3be-465e-ad91-0b4c46938047 None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:45.760833 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 325/645] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:31:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:45.760924 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d503181-d3be-465e-ad91-0b4c46938047 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:45.760924 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d503181-d3be-465e-ad91-0b4c46938047 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:45.761074 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae8bbec-4b80-42af-a454-7c3ba44d29b4 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:45.761154 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d503181-d3be-465e-ad91-0b4c46938047 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:45.783561 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 321/646] 10.222.0.42 () {56 vars in 1137 bytes} [Wed Jun 15 14:31:45 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:45.783561 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb78ed3b-4eb2-4685-8bd5-38c76a9ba897 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:45.783561 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb78ed3b-4eb2-4685-8bd5-38c76a9ba897 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:45.783561 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb78ed3b-4eb2-4685-8bd5-38c76a9ba897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:45.799596 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-eb78ed3b-4eb2-4685-8bd5-38c76a9ba897 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:45.800921 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-eb78ed3b-4eb2-4685-8bd5-38c76a9ba897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:45.849415 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:45.873050 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 322/647] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:31:45.873953 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb78ed3b-4eb2-4685-8bd5-38c76a9ba897 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:45.887004 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae8bbec-4b80-42af-a454-7c3ba44d29b4 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:45.908607 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:45.927778 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea27264-9bfa-41f5-8b84-c2b91a87ab78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:45.928600 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea27264-9bfa-41f5-8b84-c2b91a87ab78 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:45.928890 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea27264-9bfa-41f5-8b84-c2b91a87ab78 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:45.929532 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea27264-9bfa-41f5-8b84-c2b91a87ab78 None admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:45.930496 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea27264-9bfa-41f5-8b84-c2b91a87ab78 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:45.931179 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea27264-9bfa-41f5-8b84-c2b91a87ab78 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:45.933135 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea27264-9bfa-41f5-8b84-c2b91a87ab78 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:45.950896 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 326/648] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jun 15 14:31:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:45.951519 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea27264-9bfa-41f5-8b84-c2b91a87ab78 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:45.978340 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb78ed3b-4eb2-4685-8bd5-38c76a9ba897 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:46.016782 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca60722-0991-4dc1-87c7-3a75f015bdd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:46.017444 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca60722-0991-4dc1-87c7-3a75f015bdd1 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:46.017722 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca60722-0991-4dc1-87c7-3a75f015bdd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:46.017997 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca60722-0991-4dc1-87c7-3a75f015bdd1 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:46.018711 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca60722-0991-4dc1-87c7-3a75f015bdd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:46.019015 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca60722-0991-4dc1-87c7-3a75f015bdd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:46.020390 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca60722-0991-4dc1-87c7-3a75f015bdd1 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:46.036573 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 323/649] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:31:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:46.037651 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca60722-0991-4dc1-87c7-3a75f015bdd1 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:46.060885 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea27264-9bfa-41f5-8b84-c2b91a87ab78 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:46.082969 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:46.098656 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c788369-cc9d-4b2b-9db3-4759d651aa2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:46.099306 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c788369-cc9d-4b2b-9db3-4759d651aa2c None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:46.099644 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c788369-cc9d-4b2b-9db3-4759d651aa2c None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:46.099867 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c788369-cc9d-4b2b-9db3-4759d651aa2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:46.142091 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca60722-0991-4dc1-87c7-3a75f015bdd1 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:46.154393 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:46.177936 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c788369-cc9d-4b2b-9db3-4759d651aa2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:46.179398 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed629ab-2006-4075-bc51-f6159f34639e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:46.179638 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c788369-cc9d-4b2b-9db3-4759d651aa2c None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:46.180233 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed629ab-2006-4075-bc51-f6159f34639e None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:46.180518 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed629ab-2006-4075-bc51-f6159f34639e None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:46.180794 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed629ab-2006-4075-bc51-f6159f34639e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:46.194463 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed629ab-2006-4075-bc51-f6159f34639e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:31:46.205164 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-bed629ab-2006-4075-bc51-f6159f34639e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:31:46.205164 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:46.205164 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:46.205164 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:46.205164 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:46.205164 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:46.205164 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:46.205164 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:46.205164 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:46.205164 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:46.205164 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:46.205164 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:46.205164 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:31:46.205164 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:31:46.205164 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:31:46.205164 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:31:46.206087 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:31:46.206087 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:31:46.206087 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:31:46.206087 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:31:46.206087 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:46.206087 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:46.206087 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:46.206087 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:46.206087 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:46.206087 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:46.206087 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:46.206087 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:46.206087 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:46.206087 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:46.206087 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:46.206087 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:46.206901 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:46.206901 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:46.206901 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:46.206901 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:46.206901 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:31:46.206901 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:31:46.206901 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:31:46.206901 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:31:46.206901 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:31:46.206901 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:31:46.206901 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:31:46.206901 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:46.207528 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 324/650] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Jun 15 14:31:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:46.208154 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed629ab-2006-4075-bc51-f6159f34639e None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:46.296886 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 327/651] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jun 15 14:31:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:31:46.297590 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c788369-cc9d-4b2b-9db3-4759d651aa2c None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:46.325890 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:46.329517 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed629ab-2006-4075-bc51-f6159f34639e None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:46.367611 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e83a1-5abc-4c68-a0af-c9e1bcb50b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:46.368101 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e83a1-5abc-4c68-a0af-c9e1bcb50b8a None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:46.368399 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e83a1-5abc-4c68-a0af-c9e1bcb50b8a None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:46.368670 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e83a1-5abc-4c68-a0af-c9e1bcb50b8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:46.369489 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90e83a1-5abc-4c68-a0af-c9e1bcb50b8a None admin] RBAC: Adding query filter params (role.id=305a1ca2e6f740eba7dc696849e933eb, scope.project.id=3094ff8966f149bda784da6a51b1358f, user.id=0927c05c340c48e48b18754888aa9c35) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:46.369801 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90e83a1-5abc-4c68-a0af-c9e1bcb50b8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:46.371215 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90e83a1-5abc-4c68-a0af-c9e1bcb50b8a None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:46.377604 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 325/652] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jun 15 14:31:46 2022] GET /identity/v3/role_assignments?user.id=0927c05c340c48e48b18754888aa9c35&scope.project.id=3094ff8966f149bda784da6a51b1358f&role.id=305a1ca2e6f740eba7dc696849e933eb => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:46.378375 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e83a1-5abc-4c68-a0af-c9e1bcb50b8a None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:46.387561 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:46.415006 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c788369-cc9d-4b2b-9db3-4759d651aa2c None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:46.460917 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5f10a0-7f3b-49c6-92b9-a428ee57e558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:46.462133 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f10a0-7f3b-49c6-92b9-a428ee57e558 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:46.462533 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f10a0-7f3b-49c6-92b9-a428ee57e558 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:46.462903 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f10a0-7f3b-49c6-92b9-a428ee57e558 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:46.472997 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5f10a0-7f3b-49c6-92b9-a428ee57e558 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:46.474569 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5f10a0-7f3b-49c6-92b9-a428ee57e558 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:46.491334 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-ec5f10a0-7f3b-49c6-92b9-a428ee57e558 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 15 14:31:46.491334 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:46.491334 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:46.491334 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:46.491334 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:46.491334 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:46.491334 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:46.491334 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:46.491334 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:46.491334 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:46.491334 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:46.491334 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:46.491334 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:31:46.491334 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:31:46.491334 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:31:46.491334 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:31:46.492403 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:46.492403 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:46.492403 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:31:46.492403 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:31:46.492403 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:31:46.492403 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:31:46.492403 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:46.492403 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:46.492403 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:46.492403 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:46.492403 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:46.492403 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:46.492403 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:46.492403 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:46.492403 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:46.492403 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:46.492403 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:46.492403 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:46.493342 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:46.493342 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:46.493342 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:31:46.493342 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:31:46.493342 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:31:46.493342 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:31:46.493342 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:31:46.493342 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:31:46.493342 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 15 14:31:46.493342 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:46.493342 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 328/653] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jun 15 14:31:46 2022] GET /identity/v3/users/glance => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:46.494490 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5f10a0-7f3b-49c6-92b9-a428ee57e558 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:46.504802 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e83a1-5abc-4c68-a0af-c9e1bcb50b8a None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:46.542539 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d470dddd-45a8-47de-8d16-eef11805570b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:46.543568 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d470dddd-45a8-47de-8d16-eef11805570b None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:46.543902 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d470dddd-45a8-47de-8d16-eef11805570b None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:46.544214 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d470dddd-45a8-47de-8d16-eef11805570b None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:46.544973 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d470dddd-45a8-47de-8d16-eef11805570b None admin] RBAC: Adding query filter params (name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:46.545282 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d470dddd-45a8-47de-8d16-eef11805570b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:46.546643 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d470dddd-45a8-47de-8d16-eef11805570b None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:46.560744 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 326/654] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Jun 15 14:31:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:46.562372 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197bedda-073c-45f2-8cc1-095b71ee74fb None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:46.562885 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197bedda-073c-45f2-8cc1-095b71ee74fb None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:46.563166 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197bedda-073c-45f2-8cc1-095b71ee74fb None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:46.565223 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 327/655] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:31:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:46.565438 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197bedda-073c-45f2-8cc1-095b71ee74fb None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:46.598453 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:46.624368 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5f10a0-7f3b-49c6-92b9-a428ee57e558 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:46.662366 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1213b852-db47-4356-9eb0-f5084a995431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:46.663177 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1213b852-db47-4356-9eb0-f5084a995431 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:46.663457 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1213b852-db47-4356-9eb0-f5084a995431 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:46.663730 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1213b852-db47-4356-9eb0-f5084a995431 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:46.666962 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:46.676152 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1213b852-db47-4356-9eb0-f5084a995431 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:46.677716 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1213b852-db47-4356-9eb0-f5084a995431 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:46.682163 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197bedda-073c-45f2-8cc1-095b71ee74fb None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:46.695578 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-1213b852-db47-4356-9eb0-f5084a995431 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:46.695578 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:46.695578 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:46.695578 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:46.695578 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:46.695578 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:46.695578 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:46.695578 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:46.695578 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:46.695578 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:46.695578 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:46.695578 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:46.695578 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:31:46.695578 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:31:46.695578 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:31:46.696709 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:31:46.696709 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:46.696709 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:46.696709 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:46.696709 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:46.696709 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:46.696709 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:46.696709 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:46.696709 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:46.696709 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:46.696709 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:46.696709 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:46.696709 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:46.696709 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:46.696709 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:46.696709 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:46.696709 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:46.697856 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:31:46.697856 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:31:46.697856 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:46.697856 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:46.697856 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 329/656] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:31:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:46.698794 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1213b852-db47-4356-9eb0-f5084a995431 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:46.725734 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bfc997-2c18-4889-b441-698a5867426c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:46.726404 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bfc997-2c18-4889-b441-698a5867426c None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:46.726688 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bfc997-2c18-4889-b441-698a5867426c None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:46.726966 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bfc997-2c18-4889-b441-698a5867426c None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:46.727802 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bfc997-2c18-4889-b441-698a5867426c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:46.728146 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bfc997-2c18-4889-b441-698a5867426c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:46.729569 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bfc997-2c18-4889-b441-698a5867426c None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:46.752447 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 328/657] 10.222.0.42 () {56 vars in 1137 bytes} [Wed Jun 15 14:31:46 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:46.753767 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a443422-517a-47e3-98f5-8e95a82a1bb1 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:46.754059 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a443422-517a-47e3-98f5-8e95a82a1bb1 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:46.754340 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a443422-517a-47e3-98f5-8e95a82a1bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:46.770795 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-8a443422-517a-47e3-98f5-8e95a82a1bb1 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:46.772115 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-8a443422-517a-47e3-98f5-8e95a82a1bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:46.812007 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1213b852-db47-4356-9eb0-f5084a995431 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:46.856337 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 329/658] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:31:46.856337 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a443422-517a-47e3-98f5-8e95a82a1bb1 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:46.859738 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7b97d2-0177-4316-b910-f23dfd813589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:46.860825 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7b97d2-0177-4316-b910-f23dfd813589 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:46.861353 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7b97d2-0177-4316-b910-f23dfd813589 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:46.861727 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7b97d2-0177-4316-b910-f23dfd813589 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:46.870681 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:46.874758 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7b97d2-0177-4316-b910-f23dfd813589 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:46.876482 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7b97d2-0177-4316-b910-f23dfd813589 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:46.894206 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-0c7b97d2-0177-4316-b910-f23dfd813589 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:46.894206 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:46.894206 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:46.894206 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:46.894206 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:46.894206 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:46.894206 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:46.894206 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:46.894206 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:46.894206 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:46.894206 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:46.894206 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:46.894206 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:31:46.894206 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:31:46.894206 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:31:46.895131 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:31:46.895131 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:46.895131 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:46.895131 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:46.895131 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:46.895131 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:46.895131 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:46.895131 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:46.895131 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:46.895131 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:46.895131 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:46.895131 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:46.895131 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:46.895131 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:46.895131 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:46.895131 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:46.895131 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:46.896012 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:31:46.896012 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:31:46.896012 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:46.896012 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:46.896012 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 330/659] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:31:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:46.896342 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7b97d2-0177-4316-b910-f23dfd813589 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:46.945238 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:46.963545 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a443422-517a-47e3-98f5-8e95a82a1bb1 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:47.001115 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc50a3d-cc06-4d23-9146-c5fdfcfa63ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:47.001782 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc50a3d-cc06-4d23-9146-c5fdfcfa63ea None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:47.002064 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc50a3d-cc06-4d23-9146-c5fdfcfa63ea None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:47.002340 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc50a3d-cc06-4d23-9146-c5fdfcfa63ea None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:47.003062 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc50a3d-cc06-4d23-9146-c5fdfcfa63ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:47.003373 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc50a3d-cc06-4d23-9146-c5fdfcfa63ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:47.004767 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc50a3d-cc06-4d23-9146-c5fdfcfa63ea None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:47.014085 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7b97d2-0177-4316-b910-f23dfd813589 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:47.017620 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 330/660] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:31:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:47.018683 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc50a3d-cc06-4d23-9146-c5fdfcfa63ea None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:47.050904 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fc47e5-e104-49f0-8fc9-62d5fdc3e894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:47.051870 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc47e5-e104-49f0-8fc9-62d5fdc3e894 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:47.052378 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc47e5-e104-49f0-8fc9-62d5fdc3e894 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:47.052848 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc47e5-e104-49f0-8fc9-62d5fdc3e894 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:47.097301 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:47.123084 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc50a3d-cc06-4d23-9146-c5fdfcfa63ea None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:47.127662 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fc47e5-e104-49f0-8fc9-62d5fdc3e894 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:47.129079 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fc47e5-e104-49f0-8fc9-62d5fdc3e894 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:47.158028 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37567622-a186-4068-b647-86b86ccf7daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:47.158826 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37567622-a186-4068-b647-86b86ccf7daa None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:47.159104 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37567622-a186-4068-b647-86b86ccf7daa None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:47.159380 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37567622-a186-4068-b647-86b86ccf7daa None admin] PATH_INFO: `/v3/roles/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:47.171410 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37567622-a186-4068-b647-86b86ccf7daa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:31:47.172300 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:47.179521 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-37567622-a186-4068-b647-86b86ccf7daa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:31:47.179521 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:47.179521 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:47.179521 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:47.179521 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:47.179521 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:47.179521 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:47.179521 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:47.179521 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:47.179521 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:47.179521 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:47.179521 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:47.179521 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:31:47.179521 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:31:47.179521 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:31:47.179521 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:31:47.187999 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:31:47.187999 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:31:47.187999 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:31:47.187999 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:31:47.187999 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:47.187999 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:47.187999 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:47.187999 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:47.187999 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:47.187999 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:47.187999 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:47.187999 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:47.187999 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:47.187999 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:47.187999 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:47.187999 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:47.188916 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:47.188916 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:47.188916 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:47.188916 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:47.188916 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:31:47.188916 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:31:47.188916 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:31:47.188916 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:31:47.188916 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:31:47.188916 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:31:47.188916 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:31:47.188916 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:47.188916 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 331/661] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Jun 15 14:31:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:47.188916 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37567622-a186-4068-b647-86b86ccf7daa None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:47.244796 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 331/662] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jun 15 14:31:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:31:47.245528 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fc47e5-e104-49f0-8fc9-62d5fdc3e894 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:47.302802 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37567622-a186-4068-b647-86b86ccf7daa None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:47.339639 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd6383-69ca-4d93-844a-c8c29335e53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:47.340675 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd6383-69ca-4d93-844a-c8c29335e53e None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:47.341170 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd6383-69ca-4d93-844a-c8c29335e53e None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:47.341460 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd6383-69ca-4d93-844a-c8c29335e53e None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:47.342188 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fd6383-69ca-4d93-844a-c8c29335e53e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:47.342502 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fd6383-69ca-4d93-844a-c8c29335e53e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:47.344196 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fd6383-69ca-4d93-844a-c8c29335e53e None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:47.345753 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:47.356200 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fc47e5-e104-49f0-8fc9-62d5fdc3e894 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:47.362144 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 332/663] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:31:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:47.362144 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd6383-69ca-4d93-844a-c8c29335e53e None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:47.391054 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69926f5-8184-44e5-8f3a-130aaff558b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:47.391767 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69926f5-8184-44e5-8f3a-130aaff558b4 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:47.392050 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69926f5-8184-44e5-8f3a-130aaff558b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:47.392355 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69926f5-8184-44e5-8f3a-130aaff558b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:47.398880 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69926f5-8184-44e5-8f3a-130aaff558b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:47.400599 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69926f5-8184-44e5-8f3a-130aaff558b4 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:47.403543 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:47.415265 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-a69926f5-8184-44e5-8f3a-130aaff558b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:31:47.415265 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:47.415265 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:47.415265 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:47.415265 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:47.415265 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:47.415265 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:47.415265 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:47.415265 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:47.415265 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:47.415265 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:47.415265 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:47.415265 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:31:47.415265 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:31:47.415265 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:31:47.416453 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:31:47.416453 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:47.416453 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:47.416453 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:47.416453 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:47.416453 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:47.416453 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:47.416453 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:47.416453 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:47.416453 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:47.416453 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:47.416453 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:47.416453 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:47.416453 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:47.416453 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:47.416453 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:47.416453 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:47.417369 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:31:47.417369 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:31:47.417369 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:31:47.417369 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:31:47.417369 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:31:47.417369 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:31:47.417369 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:31:47.417369 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:47.417369 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 332/664] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jun 15 14:31:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:47.417850 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69926f5-8184-44e5-8f3a-130aaff558b4 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:47.486059 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd6383-69ca-4d93-844a-c8c29335e53e None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:47.523439 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0910ba98-de36-4320-9a50-a465d3abdff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:47.524394 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0910ba98-de36-4320-9a50-a465d3abdff1 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:47.524683 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0910ba98-de36-4320-9a50-a465d3abdff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:47.524985 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0910ba98-de36-4320-9a50-a465d3abdff1 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:47.525699 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0910ba98-de36-4320-9a50-a465d3abdff1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:47.526000 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0910ba98-de36-4320-9a50-a465d3abdff1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:47.527351 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0910ba98-de36-4320-9a50-a465d3abdff1 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:47.528893 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69926f5-8184-44e5-8f3a-130aaff558b4 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:47.537442 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 333/665] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Jun 15 14:31:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:47.543003 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0910ba98-de36-4320-9a50-a465d3abdff1 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:47.574168 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c400c925-1a95-40b5-bab3-53f2efae5866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:47.575175 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c400c925-1a95-40b5-bab3-53f2efae5866 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:47.575589 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c400c925-1a95-40b5-bab3-53f2efae5866 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:47.575864 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c400c925-1a95-40b5-bab3-53f2efae5866 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:47.586566 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c400c925-1a95-40b5-bab3-53f2efae5866 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:47.588072 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c400c925-1a95-40b5-bab3-53f2efae5866 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:47.605619 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-c400c925-1a95-40b5-bab3-53f2efae5866 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:47.605619 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:47.605619 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:47.605619 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:47.605619 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:47.605619 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:47.605619 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:47.605619 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:47.605619 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:47.605619 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:47.605619 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:47.605619 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:47.605619 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:31:47.605619 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:31:47.605619 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:31:47.607237 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:31:47.607237 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:47.607237 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:47.607237 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:47.607237 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:47.607237 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:47.607237 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:47.607237 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:47.607237 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:47.607237 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:47.607237 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:47.607237 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:47.607237 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:47.607237 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:47.607237 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:47.607237 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:47.607237 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:47.608174 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:31:47.608174 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:31:47.608174 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:47.608174 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:47.608410 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:47.608485 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 333/666] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:31:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:47.608855 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c400c925-1a95-40b5-bab3-53f2efae5866 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:47.659024 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0910ba98-de36-4320-9a50-a465d3abdff1 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:47.670503 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:47.695993 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a830f9b-43a6-4a7f-b5d3-3226a96e977b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:47.696823 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a830f9b-43a6-4a7f-b5d3-3226a96e977b None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:47.697111 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a830f9b-43a6-4a7f-b5d3-3226a96e977b None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:47.697384 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a830f9b-43a6-4a7f-b5d3-3226a96e977b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:47.705948 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a830f9b-43a6-4a7f-b5d3-3226a96e977b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:47.707368 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a830f9b-43a6-4a7f-b5d3-3226a96e977b None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:47.720653 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c400c925-1a95-40b5-bab3-53f2efae5866 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:47.722135 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-5a830f9b-43a6-4a7f-b5d3-3226a96e977b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 15 14:31:47.722135 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:47.722135 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:47.722135 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:47.722135 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:47.722135 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:47.722135 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:47.722135 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:47.722135 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:47.722135 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:47.722135 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:47.722135 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:47.722135 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:31:47.722135 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:31:47.722135 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:31:47.722135 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:31:47.723263 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:47.723263 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:47.723263 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:31:47.723263 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:31:47.723263 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:31:47.723263 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:31:47.723263 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:47.723263 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:47.723263 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:47.723263 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:47.723263 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:47.723263 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:47.723263 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:47.723263 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:47.723263 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:47.723263 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:47.723263 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:47.723263 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:47.724288 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:47.724288 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:47.724288 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:31:47.724288 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:31:47.724288 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:31:47.724288 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:31:47.724288 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:31:47.724288 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:31:47.724288 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 15 14:31:47.724288 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:47.724288 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 334/667] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Jun 15 14:31:47 2022] GET /identity/v3/users/neutron => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:47.724905 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a830f9b-43a6-4a7f-b5d3-3226a96e977b None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:47.760780 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156a41ee-3dfe-417c-a569-eefe436aa235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:47.761440 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156a41ee-3dfe-417c-a569-eefe436aa235 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:47.761727 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156a41ee-3dfe-417c-a569-eefe436aa235 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:47.762542 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156a41ee-3dfe-417c-a569-eefe436aa235 None admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:47.763545 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156a41ee-3dfe-417c-a569-eefe436aa235 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:47.763860 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156a41ee-3dfe-417c-a569-eefe436aa235 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:47.765322 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156a41ee-3dfe-417c-a569-eefe436aa235 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:47.779490 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 334/668] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jun 15 14:31:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:47.780696 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156a41ee-3dfe-417c-a569-eefe436aa235 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:47.834752 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a830f9b-43a6-4a7f-b5d3-3226a96e977b None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:47.852464 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:47.874311 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cb5393-b190-43fa-a39e-90e11a4351ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:47.875119 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb5393-b190-43fa-a39e-90e11a4351ec None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:47.875410 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb5393-b190-43fa-a39e-90e11a4351ec None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:47.875663 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb5393-b190-43fa-a39e-90e11a4351ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:47.887133 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cb5393-b190-43fa-a39e-90e11a4351ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:47.888721 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156a41ee-3dfe-417c-a569-eefe436aa235 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:47.889004 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cb5393-b190-43fa-a39e-90e11a4351ec None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:47.902883 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:47.906949 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-b5cb5393-b190-43fa-a39e-90e11a4351ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:47.906949 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:47.906949 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:47.906949 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:47.906949 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:47.906949 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:47.906949 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:47.906949 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:47.906949 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:47.906949 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:47.906949 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:47.906949 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:47.906949 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:31:47.906949 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:31:47.906949 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:31:47.907816 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:31:47.907816 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:47.907816 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:47.907816 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:47.907816 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:47.907816 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:47.907816 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:47.907816 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:47.907816 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:47.907816 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:47.907816 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:47.907816 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:47.907816 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:47.907816 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:47.907816 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:47.907816 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:47.907816 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:47.908765 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:31:47.908765 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:31:47.908765 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:47.908765 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:47.915383 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 335/669] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:31:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:47.915383 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cb5393-b190-43fa-a39e-90e11a4351ec None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:47.929340 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cfb38a-1d19-4e28-a28f-227e71340649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:47.930019 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cfb38a-1d19-4e28-a28f-227e71340649 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:47.930306 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cfb38a-1d19-4e28-a28f-227e71340649 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:47.930584 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cfb38a-1d19-4e28-a28f-227e71340649 None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:47.932574 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cfb38a-1d19-4e28-a28f-227e71340649 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:47.932574 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cfb38a-1d19-4e28-a28f-227e71340649 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:47.933065 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cfb38a-1d19-4e28-a28f-227e71340649 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:47.951074 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 335/670] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:31:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:47.951074 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cfb38a-1d19-4e28-a28f-227e71340649 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:48.026790 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cb5393-b190-43fa-a39e-90e11a4351ec None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:48.053236 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cfb38a-1d19-4e28-a28f-227e71340649 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:48.065306 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3e6bf3-edc6-4f97-ad91-6e7790ac8f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:48.066095 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e6bf3-edc6-4f97-ad91-6e7790ac8f28 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:48.066380 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e6bf3-edc6-4f97-ad91-6e7790ac8f28 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:48.066782 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e6bf3-edc6-4f97-ad91-6e7790ac8f28 None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:48.067619 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3e6bf3-edc6-4f97-ad91-6e7790ac8f28 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:48.067931 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3e6bf3-edc6-4f97-ad91-6e7790ac8f28 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:48.069496 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3e6bf3-edc6-4f97-ad91-6e7790ac8f28 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:48.087225 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1668bc-2a01-455c-9e56-211006c16288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:48.087881 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1668bc-2a01-455c-9e56-211006c16288 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:48.088198 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1668bc-2a01-455c-9e56-211006c16288 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:48.088479 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1668bc-2a01-455c-9e56-211006c16288 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:48.089305 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1668bc-2a01-455c-9e56-211006c16288 None admin] RBAC: Adding query filter params (role.id=305a1ca2e6f740eba7dc696849e933eb, scope.project.id=3094ff8966f149bda784da6a51b1358f, user.id=e44f6cd435be4e03bc79cb3125d71451) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:48.089626 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1668bc-2a01-455c-9e56-211006c16288 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:48.091693 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1668bc-2a01-455c-9e56-211006c16288 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:48.095160 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 336/671] 10.222.0.42 () {56 vars in 1139 bytes} [Wed Jun 15 14:31:47 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:48.095973 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3e6bf3-edc6-4f97-ad91-6e7790ac8f28 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:48.108158 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 336/672] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jun 15 14:31:47 2022] GET /identity/v3/role_assignments?user.id=e44f6cd435be4e03bc79cb3125d71451&scope.project.id=3094ff8966f149bda784da6a51b1358f&role.id=305a1ca2e6f740eba7dc696849e933eb => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:48.108158 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1668bc-2a01-455c-9e56-211006c16288 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:48.138985 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:48.150100 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:48.235241 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3e6bf3-edc6-4f97-ad91-6e7790ac8f28 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:48.242618 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1668bc-2a01-455c-9e56-211006c16288 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:48.280791 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d32cb4-4341-431f-af17-a5937bca6239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:48.281653 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d32cb4-4341-431f-af17-a5937bca6239 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:48.282099 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d32cb4-4341-431f-af17-a5937bca6239 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:48.282445 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d32cb4-4341-431f-af17-a5937bca6239 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:48.283249 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d32cb4-4341-431f-af17-a5937bca6239 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:48.283645 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d32cb4-4341-431f-af17-a5937bca6239 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:48.285162 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d32cb4-4341-431f-af17-a5937bca6239 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:48.287946 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c8f436-4c0f-4d2e-9e24-98a920761caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:48.289076 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c8f436-4c0f-4d2e-9e24-98a920761caa None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:48.289571 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c8f436-4c0f-4d2e-9e24-98a920761caa None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:48.290052 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c8f436-4c0f-4d2e-9e24-98a920761caa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:48.298870 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c8f436-4c0f-4d2e-9e24-98a920761caa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:48.299085 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 337/673] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:31:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:48.299727 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d32cb4-4341-431f-af17-a5937bca6239 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:48.301262 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c8f436-4c0f-4d2e-9e24-98a920761caa None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:48.318245 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-28c8f436-4c0f-4d2e-9e24-98a920761caa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:31:48.318245 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:48.318245 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:48.318245 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:48.318245 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:48.318245 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:48.318245 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:48.318245 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:48.318245 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:48.318245 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:48.318245 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:48.318245 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:48.318245 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:31:48.318245 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:31:48.318245 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:31:48.319199 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:31:48.319199 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:48.319199 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:48.319199 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:48.319199 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:48.319199 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:48.319199 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:48.319199 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:48.319199 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:48.319199 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:48.319199 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:48.319199 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:48.319199 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:48.319199 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:48.319199 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:48.319199 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:48.319199 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:48.320285 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:31:48.320285 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:31:48.320285 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:31:48.320285 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:31:48.320285 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:31:48.320285 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:31:48.320285 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:31:48.320285 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:48.325314 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 337/674] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jun 15 14:31:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:48.325314 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c8f436-4c0f-4d2e-9e24-98a920761caa None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:48.412660 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:48.426624 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d32cb4-4341-431f-af17-a5937bca6239 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:48.456485 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c8f436-4c0f-4d2e-9e24-98a920761caa None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:48.470241 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7dbd9d-0b19-4282-b912-0ffb1a830da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:48.470904 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7dbd9d-0b19-4282-b912-0ffb1a830da4 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:48.471237 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:48.471344 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7dbd9d-0b19-4282-b912-0ffb1a830da4 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:48.472055 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7dbd9d-0b19-4282-b912-0ffb1a830da4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:48.497323 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6b88a0-2d8b-4630-8504-84b24b717921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:48.498504 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6b88a0-2d8b-4630-8504-84b24b717921 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:48.499108 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6b88a0-2d8b-4630-8504-84b24b717921 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:48.499582 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6b88a0-2d8b-4630-8504-84b24b717921 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:48.507761 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6b88a0-2d8b-4630-8504-84b24b717921 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:48.509815 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6b88a0-2d8b-4630-8504-84b24b717921 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:48.527088 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-ec6b88a0-2d8b-4630-8504-84b24b717921 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 15 14:31:48.527088 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:48.527088 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:48.527088 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:48.527088 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:48.527088 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:48.527088 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:48.527088 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:48.527088 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:48.527088 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:48.527088 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:48.527088 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:48.527088 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:31:48.527088 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:31:48.527088 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:31:48.527088 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:31:48.529028 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:48.529028 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:48.529028 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:31:48.529028 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:31:48.529028 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:31:48.529028 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:31:48.529028 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:48.529028 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:48.529028 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:48.529028 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:48.529028 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:48.529028 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:48.529028 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:48.529028 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:48.529028 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:48.529028 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:48.529028 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:48.529028 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:48.529982 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:48.529982 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:48.529982 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:31:48.529982 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:31:48.529982 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:31:48.529982 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:31:48.529982 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:31:48.529982 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:31:48.529982 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 15 14:31:48.529982 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:48.532953 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 338/675] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jun 15 14:31:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:48.534173 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6b88a0-2d8b-4630-8504-84b24b717921 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:48.566794 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7dbd9d-0b19-4282-b912-0ffb1a830da4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:48.568693 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7dbd9d-0b19-4282-b912-0ffb1a830da4 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:48.660921 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6b88a0-2d8b-4630-8504-84b24b717921 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:48.672434 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:48.687441 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 338/676] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jun 15 14:31:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:31:48.688294 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7dbd9d-0b19-4282-b912-0ffb1a830da4 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:48.704244 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf3f012-d49c-4ac6-aa50-88522e87a3ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:48.705573 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf3f012-d49c-4ac6-aa50-88522e87a3ce None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:48.706064 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf3f012-d49c-4ac6-aa50-88522e87a3ce None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:48.706555 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf3f012-d49c-4ac6-aa50-88522e87a3ce None admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:48.707536 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf3f012-d49c-4ac6-aa50-88522e87a3ce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:48.708042 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf3f012-d49c-4ac6-aa50-88522e87a3ce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:48.709756 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf3f012-d49c-4ac6-aa50-88522e87a3ce None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:48.722576 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 339/677] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jun 15 14:31:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:48.723938 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b0f2b6-52a6-4890-9bb2-be9de4392c50 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:48.724325 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b0f2b6-52a6-4890-9bb2-be9de4392c50 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:48.724674 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b0f2b6-52a6-4890-9bb2-be9de4392c50 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:48.728152 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 340/678] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:31:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:48.728152 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b0f2b6-52a6-4890-9bb2-be9de4392c50 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:48.749113 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:48.809818 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7dbd9d-0b19-4282-b912-0ffb1a830da4 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:48.838373 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b0f2b6-52a6-4890-9bb2-be9de4392c50 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:48.845752 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f361040-95a2-4be7-873b-3aa1a81e1694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:48.846420 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f361040-95a2-4be7-873b-3aa1a81e1694 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:48.846705 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f361040-95a2-4be7-873b-3aa1a81e1694 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:48.846982 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f361040-95a2-4be7-873b-3aa1a81e1694 None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:48.847814 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f361040-95a2-4be7-873b-3aa1a81e1694 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:48.848156 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f361040-95a2-4be7-873b-3aa1a81e1694 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:48.849578 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f361040-95a2-4be7-873b-3aa1a81e1694 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:48.874308 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 339/679] 10.222.0.42 () {56 vars in 1135 bytes} [Wed Jun 15 14:31:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:48.875218 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f361040-95a2-4be7-873b-3aa1a81e1694 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:48.875851 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa05aeb-3f27-4d83-8eda-46ad631f60b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:48.876747 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa05aeb-3f27-4d83-8eda-46ad631f60b5 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:48.877031 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa05aeb-3f27-4d83-8eda-46ad631f60b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:48.877307 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa05aeb-3f27-4d83-8eda-46ad631f60b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:48.889100 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa05aeb-3f27-4d83-8eda-46ad631f60b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:48.890549 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa05aeb-3f27-4d83-8eda-46ad631f60b5 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:48.909308 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-8fa05aeb-3f27-4d83-8eda-46ad631f60b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:48.909308 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:48.909308 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:48.909308 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:48.909308 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:48.909308 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:48.909308 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:48.909308 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:48.909308 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:48.909308 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:48.909308 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:48.909308 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:48.909308 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:31:48.909308 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:31:48.909308 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:31:48.910420 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:31:48.910420 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:48.910420 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:48.910420 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:48.910420 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:48.910420 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:48.910420 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:48.910420 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:48.910420 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:48.910420 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:48.910420 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:48.910420 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:48.910420 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:48.910420 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:48.910420 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:48.910420 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:48.910420 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:48.911313 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:31:48.911313 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:31:48.911313 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:48.911313 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:48.911313 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 341/680] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:31:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:48.911690 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270d2362-67e6-45f4-b0e7-6cde4aea5907 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:48.911977 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270d2362-67e6-45f4-b0e7-6cde4aea5907 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:48.912294 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270d2362-67e6-45f4-b0e7-6cde4aea5907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:48.930223 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-270d2362-67e6-45f4-b0e7-6cde4aea5907 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:48.931396 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-270d2362-67e6-45f4-b0e7-6cde4aea5907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:48.933473 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:48.993284 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f361040-95a2-4be7-873b-3aa1a81e1694 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:48.995938 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:49.012206 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 342/681] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:31:49.012206 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270d2362-67e6-45f4-b0e7-6cde4aea5907 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:49.044725 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bd0b7b-f692-4c58-a160-b657012bf954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:49.045433 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bd0b7b-f692-4c58-a160-b657012bf954 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:49.045710 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bd0b7b-f692-4c58-a160-b657012bf954 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:49.046013 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bd0b7b-f692-4c58-a160-b657012bf954 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:49.046837 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bd0b7b-f692-4c58-a160-b657012bf954 None admin] RBAC: Adding query filter params (role.id=305a1ca2e6f740eba7dc696849e933eb, scope.project.id=3094ff8966f149bda784da6a51b1358f, user.id=4f18c57f27784694935bfb0bcea48a78) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:49.047147 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bd0b7b-f692-4c58-a160-b657012bf954 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:49.048589 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bd0b7b-f692-4c58-a160-b657012bf954 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:49.062331 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 340/682] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jun 15 14:31:48 2022] GET /identity/v3/role_assignments?user.id=4f18c57f27784694935bfb0bcea48a78&scope.project.id=3094ff8966f149bda784da6a51b1358f&role.id=305a1ca2e6f740eba7dc696849e933eb => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:49.062331 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bd0b7b-f692-4c58-a160-b657012bf954 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:49.128932 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270d2362-67e6-45f4-b0e7-6cde4aea5907 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:49.182318 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efebf03c-6310-4d42-bb9b-35998ef6d667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:49.183335 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efebf03c-6310-4d42-bb9b-35998ef6d667 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:49.183735 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efebf03c-6310-4d42-bb9b-35998ef6d667 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:49.184715 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efebf03c-6310-4d42-bb9b-35998ef6d667 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:49.190572 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bd0b7b-f692-4c58-a160-b657012bf954 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:49.202124 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:49.236905 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e95e579-1435-4054-a8bb-e03bb462818b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:49.237581 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e95e579-1435-4054-a8bb-e03bb462818b None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:49.238348 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e95e579-1435-4054-a8bb-e03bb462818b None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:49.238611 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e95e579-1435-4054-a8bb-e03bb462818b None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:49.239347 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e95e579-1435-4054-a8bb-e03bb462818b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:49.239660 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e95e579-1435-4054-a8bb-e03bb462818b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:49.241737 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e95e579-1435-4054-a8bb-e03bb462818b None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:49.260167 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 341/683] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:31:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:49.260167 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e95e579-1435-4054-a8bb-e03bb462818b None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:49.281445 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:49.285806 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efebf03c-6310-4d42-bb9b-35998ef6d667 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:49.287445 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efebf03c-6310-4d42-bb9b-35998ef6d667 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:49.385336 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e95e579-1435-4054-a8bb-e03bb462818b None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:49.407056 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 343/684] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jun 15 14:31:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:31:49.408001 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efebf03c-6310-4d42-bb9b-35998ef6d667 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:49.438144 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29da1466-d450-4773-825a-916bc287b382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:49.439266 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29da1466-d450-4773-825a-916bc287b382 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:49.439796 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29da1466-d450-4773-825a-916bc287b382 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:49.440072 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29da1466-d450-4773-825a-916bc287b382 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:49.455964 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29da1466-d450-4773-825a-916bc287b382 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:31:49.467912 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-29da1466-d450-4773-825a-916bc287b382 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:31:49.467912 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:49.467912 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:49.467912 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:49.467912 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:49.467912 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:49.467912 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:49.467912 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:49.467912 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:49.467912 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:49.467912 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:49.467912 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:49.467912 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:31:49.467912 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:31:49.467912 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:31:49.467912 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:31:49.468954 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:31:49.468954 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:31:49.468954 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:31:49.468954 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:31:49.468954 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:49.468954 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:49.468954 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:49.468954 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:49.468954 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:49.468954 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:49.468954 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:49.468954 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:49.468954 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:49.468954 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:49.468954 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:49.468954 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:49.469802 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:49.469802 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:49.469802 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:49.469802 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:49.469802 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:31:49.469802 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:31:49.469802 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:31:49.469802 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:31:49.469802 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:31:49.469802 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:31:49.469802 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:31:49.469802 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:49.470494 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 342/685] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jun 15 14:31:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:49.475356 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29da1466-d450-4773-825a-916bc287b382 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:49.515638 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:49.522921 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:49.529669 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efebf03c-6310-4d42-bb9b-35998ef6d667 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:49.584344 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975e5a9a-f3e5-4ea5-b848-525f0e1fbefb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:49.585524 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975e5a9a-f3e5-4ea5-b848-525f0e1fbefb None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:49.585895 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975e5a9a-f3e5-4ea5-b848-525f0e1fbefb None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:49.586708 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975e5a9a-f3e5-4ea5-b848-525f0e1fbefb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:49.596736 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975e5a9a-f3e5-4ea5-b848-525f0e1fbefb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:49.598950 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975e5a9a-f3e5-4ea5-b848-525f0e1fbefb None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:49.606962 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29da1466-d450-4773-825a-916bc287b382 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:49.616735 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-975e5a9a-f3e5-4ea5-b848-525f0e1fbefb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:31:49.616735 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:49.616735 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:49.616735 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:49.616735 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:49.616735 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:49.616735 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:49.616735 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:49.616735 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:49.616735 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:49.616735 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:49.616735 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:49.616735 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:31:49.616735 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:31:49.616735 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:31:49.617802 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:31:49.617802 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:49.617802 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:49.617802 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:49.617802 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:49.617802 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:49.617802 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:49.617802 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:49.617802 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:49.617802 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:49.617802 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:49.617802 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:49.617802 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:49.617802 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:49.617802 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:49.617802 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:49.617802 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:49.618681 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:31:49.618681 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:31:49.618681 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:31:49.618681 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:31:49.618681 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:31:49.618681 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:31:49.618681 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:31:49.618681 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:49.624210 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 344/686] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jun 15 14:31:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:49.625278 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975e5a9a-f3e5-4ea5-b848-525f0e1fbefb None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:49.657864 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf57d214-4c6a-4171-ac20-110707fa3c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:49.658728 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf57d214-4c6a-4171-ac20-110707fa3c50 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:49.658998 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf57d214-4c6a-4171-ac20-110707fa3c50 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:49.659278 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf57d214-4c6a-4171-ac20-110707fa3c50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:49.672021 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf57d214-4c6a-4171-ac20-110707fa3c50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:49.673660 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf57d214-4c6a-4171-ac20-110707fa3c50 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:49.692400 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-cf57d214-4c6a-4171-ac20-110707fa3c50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:49.692400 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:49.692400 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:49.692400 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:49.692400 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:49.692400 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:49.692400 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:49.692400 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:49.692400 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:49.692400 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:49.692400 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:49.692400 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:49.692400 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:31:49.692400 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:31:49.692400 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:31:49.693344 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:31:49.693344 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:49.693344 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:49.693344 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:49.693344 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:49.693344 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:49.693344 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:49.693344 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:49.693344 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:49.693344 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:49.693344 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:49.693344 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:49.693344 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:49.693344 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:49.693344 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:49.693344 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:49.693344 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:49.694221 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:31:49.694221 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:31:49.694221 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:49.694221 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:49.698347 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 343/687] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:31:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:49.699192 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf57d214-4c6a-4171-ac20-110707fa3c50 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:49.758295 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975e5a9a-f3e5-4ea5-b848-525f0e1fbefb None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:49.786522 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:49.798996 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9018f071-c410-4f6b-a360-864ec09311b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:49.805024 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9018f071-c410-4f6b-a360-864ec09311b9 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:49.805024 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9018f071-c410-4f6b-a360-864ec09311b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:49.805024 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9018f071-c410-4f6b-a360-864ec09311b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:49.805024 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9018f071-c410-4f6b-a360-864ec09311b9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:49.805024 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9018f071-c410-4f6b-a360-864ec09311b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:49.805024 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9018f071-c410-4f6b-a360-864ec09311b9 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:49.806930 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf57d214-4c6a-4171-ac20-110707fa3c50 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:49.813124 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 345/688] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jun 15 14:31:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:49.814210 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9018f071-c410-4f6b-a360-864ec09311b9 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:49.838712 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:49.843157 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaf79de-723f-4387-9af1-7c27870abe5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:49.843818 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaf79de-723f-4387-9af1-7c27870abe5b None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:49.844102 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaf79de-723f-4387-9af1-7c27870abe5b None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:49.844408 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaf79de-723f-4387-9af1-7c27870abe5b None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:49.845202 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaf79de-723f-4387-9af1-7c27870abe5b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:49.845515 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaf79de-723f-4387-9af1-7c27870abe5b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:49.847076 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaf79de-723f-4387-9af1-7c27870abe5b None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:49.866511 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 344/689] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jun 15 14:31:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:49.866511 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaf79de-723f-4387-9af1-7c27870abe5b None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:49.935775 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9018f071-c410-4f6b-a360-864ec09311b9 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:49.973792 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72bddca-9f51-4993-85d3-5eba634b5e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:49.974767 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaf79de-723f-4387-9af1-7c27870abe5b None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:49.975154 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72bddca-9f51-4993-85d3-5eba634b5e82 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:49.975629 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72bddca-9f51-4993-85d3-5eba634b5e82 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:49.976230 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72bddca-9f51-4993-85d3-5eba634b5e82 None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:49.977352 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72bddca-9f51-4993-85d3-5eba634b5e82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:49.978037 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72bddca-9f51-4993-85d3-5eba634b5e82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:49.979615 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72bddca-9f51-4993-85d3-5eba634b5e82 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:49.994469 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 346/690] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:31:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:49.995619 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72bddca-9f51-4993-85d3-5eba634b5e82 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:50.021238 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf4d880-b129-4eee-9b06-8b923aa23b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:50.022035 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf4d880-b129-4eee-9b06-8b923aa23b13 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:50.022316 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf4d880-b129-4eee-9b06-8b923aa23b13 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:50.022588 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf4d880-b129-4eee-9b06-8b923aa23b13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:50.032235 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:50.034995 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf4d880-b129-4eee-9b06-8b923aa23b13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:50.036595 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf4d880-b129-4eee-9b06-8b923aa23b13 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:50.056198 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-caf4d880-b129-4eee-9b06-8b923aa23b13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:50.056198 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:50.056198 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:50.056198 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:50.056198 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:50.056198 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:50.056198 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:50.056198 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:50.056198 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:50.056198 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:50.056198 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:50.056198 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:50.056198 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:31:50.056198 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:31:50.056198 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:31:50.057098 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:31:50.057098 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:50.057098 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:50.057098 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:50.057098 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:50.057098 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:50.057098 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:50.057098 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:50.057098 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:50.057098 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:50.057098 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:50.057098 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:50.057098 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:50.057098 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:50.057098 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:50.057098 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:50.057098 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:50.058005 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:31:50.058005 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:31:50.058005 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:50.058005 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:50.058005 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 345/691] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:31:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:50.059356 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf4d880-b129-4eee-9b06-8b923aa23b13 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:50.091161 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:50.111298 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72bddca-9f51-4993-85d3-5eba634b5e82 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:50.147700 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96618cdc-a8aa-447c-bc99-eea1e0f6cfb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:50.148728 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96618cdc-a8aa-447c-bc99-eea1e0f6cfb6 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:50.149522 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96618cdc-a8aa-447c-bc99-eea1e0f6cfb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:50.150207 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96618cdc-a8aa-447c-bc99-eea1e0f6cfb6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:50.151224 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96618cdc-a8aa-447c-bc99-eea1e0f6cfb6 None admin] RBAC: Adding query filter params (role.id=305a1ca2e6f740eba7dc696849e933eb, scope.project.id=3094ff8966f149bda784da6a51b1358f, user.id=c739b61c8a8b453e979bda720bafebb1) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:50.151746 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96618cdc-a8aa-447c-bc99-eea1e0f6cfb6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:50.153442 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96618cdc-a8aa-447c-bc99-eea1e0f6cfb6 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:50.167886 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 347/692] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jun 15 14:31:49 2022] GET /identity/v3/role_assignments?user.id=c739b61c8a8b453e979bda720bafebb1&scope.project.id=3094ff8966f149bda784da6a51b1358f&role.id=305a1ca2e6f740eba7dc696849e933eb => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:50.168927 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf4d880-b129-4eee-9b06-8b923aa23b13 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:50.170138 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96618cdc-a8aa-447c-bc99-eea1e0f6cfb6 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:50.208039 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810a99e0-939f-41cb-b588-1f9c32ed52b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:50.208882 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810a99e0-939f-41cb-b588-1f9c32ed52b9 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:50.209165 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810a99e0-939f-41cb-b588-1f9c32ed52b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:50.209441 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810a99e0-939f-41cb-b588-1f9c32ed52b9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:50.217942 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810a99e0-939f-41cb-b588-1f9c32ed52b9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:50.219478 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810a99e0-939f-41cb-b588-1f9c32ed52b9 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:50.235752 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-810a99e0-939f-41cb-b588-1f9c32ed52b9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:31:50.235752 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:50.235752 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:50.235752 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:50.235752 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:50.235752 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:50.235752 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:50.235752 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:50.235752 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:50.235752 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:50.235752 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:50.235752 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:50.235752 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:31:50.235752 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:31:50.235752 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:31:50.236732 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:31:50.236732 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:50.236732 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:50.236732 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:50.236732 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:50.236732 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:50.236732 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:50.236732 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:50.236732 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:50.236732 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:50.236732 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:50.236732 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:50.236732 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:50.236732 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:50.236732 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:50.236732 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:50.236732 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:50.237644 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:31:50.237644 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:31:50.237644 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:31:50.237644 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:31:50.237644 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:31:50.237644 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:31:50.237644 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:31:50.237644 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:50.237644 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 346/693] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jun 15 14:31:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:50.243053 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810a99e0-939f-41cb-b588-1f9c32ed52b9 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:50.307302 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:50.366001 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810a99e0-939f-41cb-b588-1f9c32ed52b9 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:50.410386 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96618cdc-a8aa-447c-bc99-eea1e0f6cfb6 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:50.411822 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eac2a48-e5f5-433f-a9e3-83214930dcf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:50.412545 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eac2a48-e5f5-433f-a9e3-83214930dcf0 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:50.412817 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eac2a48-e5f5-433f-a9e3-83214930dcf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:50.413095 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eac2a48-e5f5-433f-a9e3-83214930dcf0 None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:50.413901 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eac2a48-e5f5-433f-a9e3-83214930dcf0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:50.414216 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eac2a48-e5f5-433f-a9e3-83214930dcf0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:50.415642 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eac2a48-e5f5-433f-a9e3-83214930dcf0 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:50.434451 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 347/694] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jun 15 14:31:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:31:50.440552 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eac2a48-e5f5-433f-a9e3-83214930dcf0 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:50.459531 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabcf8f2-7796-4cde-bb3e-ba16c9f9c42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:50.460786 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabcf8f2-7796-4cde-bb3e-ba16c9f9c42b None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:50.461309 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabcf8f2-7796-4cde-bb3e-ba16c9f9c42b None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:50.461806 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabcf8f2-7796-4cde-bb3e-ba16c9f9c42b None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:50.462932 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabcf8f2-7796-4cde-bb3e-ba16c9f9c42b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:50.463442 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabcf8f2-7796-4cde-bb3e-ba16c9f9c42b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:50.465574 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabcf8f2-7796-4cde-bb3e-ba16c9f9c42b None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:50.479285 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 348/695] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:31:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:50.486887 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabcf8f2-7796-4cde-bb3e-ba16c9f9c42b None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:50.491888 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:50.572427 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eac2a48-e5f5-433f-a9e3-83214930dcf0 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:50.586502 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:50.612125 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce684a3-dca9-4048-bd7a-9d6abb068b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:50.612800 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce684a3-dca9-4048-bd7a-9d6abb068b08 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:50.613086 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce684a3-dca9-4048-bd7a-9d6abb068b08 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:50.613365 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce684a3-dca9-4048-bd7a-9d6abb068b08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:50.614228 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce684a3-dca9-4048-bd7a-9d6abb068b08 None admin] RBAC: Adding query filter params (role.id=305a1ca2e6f740eba7dc696849e933eb, scope.project.id=3094ff8966f149bda784da6a51b1358f, user.id=c04dc5832333498e96a7e02bbfb9484c) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:50.614549 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce684a3-dca9-4048-bd7a-9d6abb068b08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:50.616249 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce684a3-dca9-4048-bd7a-9d6abb068b08 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:50.618966 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabcf8f2-7796-4cde-bb3e-ba16c9f9c42b None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:50.623365 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 348/696] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jun 15 14:31:50 2022] GET /identity/v3/role_assignments?user.id=c04dc5832333498e96a7e02bbfb9484c&scope.project.id=3094ff8966f149bda784da6a51b1358f&role.id=305a1ca2e6f740eba7dc696849e933eb => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:50.624792 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e73be6b-cd53-43e2-8e9e-2334c88375ae None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:50.625089 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e73be6b-cd53-43e2-8e9e-2334c88375ae None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:50.625418 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e73be6b-cd53-43e2-8e9e-2334c88375ae None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:50.630729 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 349/697] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:31:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:50.632492 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee05ea1b-a06f-4836-b020-07a1b9e324ca None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:50.632789 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee05ea1b-a06f-4836-b020-07a1b9e324ca None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:50.633072 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee05ea1b-a06f-4836-b020-07a1b9e324ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:50.668276 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-ee05ea1b-a06f-4836-b020-07a1b9e324ca None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:50.669541 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-ee05ea1b-a06f-4836-b020-07a1b9e324ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:50.680625 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f1c8de-0608-4928-a54c-520c135abc94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:50.681934 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f1c8de-0608-4928-a54c-520c135abc94 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:50.682398 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f1c8de-0608-4928-a54c-520c135abc94 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:50.682822 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f1c8de-0608-4928-a54c-520c135abc94 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:50.706186 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f1c8de-0608-4928-a54c-520c135abc94 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:50.707657 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f1c8de-0608-4928-a54c-520c135abc94 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:50.734437 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-d0f1c8de-0608-4928-a54c-520c135abc94 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:31:50.734437 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:50.734437 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:50.734437 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:50.734437 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:50.734437 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:50.734437 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:50.734437 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:50.734437 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:50.734437 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:50.734437 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:50.734437 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:50.734437 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:31:50.734437 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:31:50.734437 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:31:50.734437 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:31:50.735665 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:50.735665 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:50.735665 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:31:50.735665 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:31:50.735665 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:31:50.735665 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:31:50.735665 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:50.735665 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:50.735665 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:50.735665 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:50.735665 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:50.735665 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:50.735665 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:50.735665 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:50.735665 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:50.735665 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:50.735665 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:50.735665 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:50.737027 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:50.737027 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:50.737027 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:31:50.737027 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:31:50.737027 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:31:50.737027 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:31:50.737027 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:31:50.737027 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:31:50.737027 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:31:50.737027 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:50.737027 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 349/698] 10.222.0.42 () {56 vars in 1093 bytes} [Wed Jun 15 14:31:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:50.742852 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f1c8de-0608-4928-a54c-520c135abc94 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:50.811634 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 350/699] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:31:50.833244 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee05ea1b-a06f-4836-b020-07a1b9e324ca None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:50.850906 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:50.897972 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f1c8de-0608-4928-a54c-520c135abc94 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:50.945835 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54336d04-5806-4be6-96b5-446f940b3356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:50.946634 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54336d04-5806-4be6-96b5-446f940b3356 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:50.946993 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54336d04-5806-4be6-96b5-446f940b3356 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:50.947344 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54336d04-5806-4be6-96b5-446f940b3356 None admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:50.949075 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54336d04-5806-4be6-96b5-446f940b3356 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:50.949075 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54336d04-5806-4be6-96b5-446f940b3356 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:50.951681 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54336d04-5806-4be6-96b5-446f940b3356 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:50.956993 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee05ea1b-a06f-4836-b020-07a1b9e324ca None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:50.968627 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:50.974301 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 350/700] 10.222.0.42 () {56 vars in 1133 bytes} [Wed Jun 15 14:31:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:50.979636 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54336d04-5806-4be6-96b5-446f940b3356 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:51.006964 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4ca9a4-01a5-4b12-ab11-b7281fa9a03d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:51.007824 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ca9a4-01a5-4b12-ab11-b7281fa9a03d None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:51.008187 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ca9a4-01a5-4b12-ab11-b7281fa9a03d None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:51.008703 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ca9a4-01a5-4b12-ab11-b7281fa9a03d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:51.009126 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4ca9a4-01a5-4b12-ab11-b7281fa9a03d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:51.010487 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4ca9a4-01a5-4b12-ab11-b7281fa9a03d None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:51.033699 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-4a4ca9a4-01a5-4b12-ab11-b7281fa9a03d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 14:31:51.033699 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:51.033699 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:31:51.033699 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:31:51.033699 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:31:51.033699 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:31:51.033699 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:31:51.033699 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:31:51.033699 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 14:31:51.033699 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:51.033699 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:31:51.033699 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:51.033699 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:51.033699 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:51.033699 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:51.033699 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:51.034743 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:51.034743 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:51.034743 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:51.034743 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:51.034743 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:51.034743 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:51.034743 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:51.034743 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:31:51.034743 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:31:51.034743 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:31:51.034743 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:31:51.034743 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:51.034743 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:51.034743 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:51.034743 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:51.034743 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:51.035600 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:51.035600 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:51.035600 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:51.035600 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:51.035600 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:51.035600 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:51.035600 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:51.035600 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:51.035600 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:51.035600 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:51.035600 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:51.035600 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:31:51.035600 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:31:51.035600 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 14:31:51.035600 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:51.035600 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 351/701] 10.222.0.42 () {56 vars in 1129 bytes} [Wed Jun 15 14:31:50 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:31:51.040721 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4ca9a4-01a5-4b12-ab11-b7281fa9a03d None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:51.116833 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54336d04-5806-4be6-96b5-446f940b3356 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:51.148236 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:51.163366 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c342046a-83f9-4f5c-87ac-448a82c625b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:51.164740 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c342046a-83f9-4f5c-87ac-448a82c625b8 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:51.186223 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c342046a-83f9-4f5c-87ac-448a82c625b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:51.186223 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c342046a-83f9-4f5c-87ac-448a82c625b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:51.187381 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4ca9a4-01a5-4b12-ab11-b7281fa9a03d None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:51.230165 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e88849-10d1-42c9-b89e-06e2344881eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:51.230946 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e88849-10d1-42c9-b89e-06e2344881eb None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:51.231286 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e88849-10d1-42c9-b89e-06e2344881eb None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:51.231582 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e88849-10d1-42c9-b89e-06e2344881eb None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:51.232740 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e88849-10d1-42c9-b89e-06e2344881eb None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:51.233051 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e88849-10d1-42c9-b89e-06e2344881eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:51.234440 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e88849-10d1-42c9-b89e-06e2344881eb None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:51.245610 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 352/702] 10.222.0.42 () {56 vars in 1124 bytes} [Wed Jun 15 14:31:51 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:51.247558 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e88849-10d1-42c9-b89e-06e2344881eb None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:51.263062 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c342046a-83f9-4f5c-87ac-448a82c625b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:51.264689 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c342046a-83f9-4f5c-87ac-448a82c625b8 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:51.286926 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:51.373483 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 351/703] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jun 15 14:31:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:31:51.378871 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c342046a-83f9-4f5c-87ac-448a82c625b8 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:51.386618 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e88849-10d1-42c9-b89e-06e2344881eb None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:51.404695 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:51.447657 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1415af-92e6-4135-aebe-fa81f5d7e792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:51.449135 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1415af-92e6-4135-aebe-fa81f5d7e792 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:51.450208 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1415af-92e6-4135-aebe-fa81f5d7e792 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:51.450748 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1415af-92e6-4135-aebe-fa81f5d7e792 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:51.457261 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1415af-92e6-4135-aebe-fa81f5d7e792 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:51.457895 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1415af-92e6-4135-aebe-fa81f5d7e792 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:51.459662 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1415af-92e6-4135-aebe-fa81f5d7e792 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:51.472779 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 353/704] 10.222.0.42 () {56 vars in 1124 bytes} [Wed Jun 15 14:31:51 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:51.489738 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c342046a-83f9-4f5c-87ac-448a82c625b8 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:51.533081 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe2f2f0-740f-4a43-92e0-96eb673ba4ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:51.533973 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe2f2f0-740f-4a43-92e0-96eb673ba4ba None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:51.534309 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe2f2f0-740f-4a43-92e0-96eb673ba4ba None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:51.535067 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe2f2f0-740f-4a43-92e0-96eb673ba4ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:51.556623 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe2f2f0-740f-4a43-92e0-96eb673ba4ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:51.558015 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe2f2f0-740f-4a43-92e0-96eb673ba4ba None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:51.573765 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b45bce-9268-4d79-9973-911201180a8e None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:51.574120 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b45bce-9268-4d79-9973-911201180a8e None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:51.574570 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b45bce-9268-4d79-9973-911201180a8e None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:51.575951 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 354/705] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:31:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:51.588171 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-bbe2f2f0-740f-4a43-92e0-96eb673ba4ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:51.588171 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:51.588171 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:51.588171 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:51.588171 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:51.588171 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:51.588171 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:51.588171 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:51.588171 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:51.588171 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:51.588171 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:51.588171 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:51.588171 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:31:51.588171 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:31:51.588171 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:31:51.589165 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:31:51.589165 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:51.589165 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:51.589165 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:51.589165 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:51.589165 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:51.589165 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:51.589165 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:51.589165 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:51.589165 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:51.589165 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:51.589165 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:51.589165 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:51.589165 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:51.589165 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:51.589165 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:51.589165 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:51.591421 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:31:51.591421 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:31:51.591421 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:51.591421 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:51.591661 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617302a4-8208-475d-a718-e0bfce0ec491 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:51.591735 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 352/706] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:31:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:31:51.591803 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617302a4-8208-475d-a718-e0bfce0ec491 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:51.591803 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617302a4-8208-475d-a718-e0bfce0ec491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:51.595808 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe2f2f0-740f-4a43-92e0-96eb673ba4ba None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:51.645409 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-617302a4-8208-475d-a718-e0bfce0ec491 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:51.646578 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-617302a4-8208-475d-a718-e0bfce0ec491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:51.684636 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:51.714852 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe2f2f0-740f-4a43-92e0-96eb673ba4ba None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:51.760660 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 355/707] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:31:51.763253 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746a1e61-ca7e-45c5-bde5-b5139ddaab4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:51.764246 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a1e61-ca7e-45c5-bde5-b5139ddaab4d None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:51.764739 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a1e61-ca7e-45c5-bde5-b5139ddaab4d None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:51.765219 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a1e61-ca7e-45c5-bde5-b5139ddaab4d None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:51.766148 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746a1e61-ca7e-45c5-bde5-b5139ddaab4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:51.766674 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746a1e61-ca7e-45c5-bde5-b5139ddaab4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:51.768331 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746a1e61-ca7e-45c5-bde5-b5139ddaab4d None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:51.783944 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 353/708] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:31:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:31:51.788959 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617302a4-8208-475d-a718-e0bfce0ec491 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:51.790602 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746a1e61-ca7e-45c5-bde5-b5139ddaab4d None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:51.820054 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:51.916144 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617302a4-8208-475d-a718-e0bfce0ec491 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:51.954848 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a2bfe0-d471-40b8-b2d8-fe40303490bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:51.955687 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a2bfe0-d471-40b8-b2d8-fe40303490bc None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:51.960117 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a2bfe0-d471-40b8-b2d8-fe40303490bc None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:51.960117 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a2bfe0-d471-40b8-b2d8-fe40303490bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:51.963252 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a2bfe0-d471-40b8-b2d8-fe40303490bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:51.965496 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a2bfe0-d471-40b8-b2d8-fe40303490bc None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:51.970072 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746a1e61-ca7e-45c5-bde5-b5139ddaab4d None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:51.979847 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-f4a2bfe0-d471-40b8-b2d8-fe40303490bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:31:51.979847 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:51.979847 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:51.979847 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:51.979847 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:51.979847 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:51.979847 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:51.979847 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:51.979847 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:51.979847 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:51.979847 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:51.979847 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:51.979847 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:31:51.979847 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:31:51.979847 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:31:51.987207 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:31:51.987207 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:51.987207 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:51.987207 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:51.987207 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:51.987207 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:51.987207 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:51.987207 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:51.987207 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:51.987207 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:51.987207 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:51.987207 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:51.987207 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:51.987207 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:51.987207 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:51.987207 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:51.987207 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:51.988131 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:31:51.988131 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:31:51.988131 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:31:51.988131 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:31:51.988131 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:31:51.988131 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:31:51.988131 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:31:51.988131 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:51.988131 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 356/709] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jun 15 14:31:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:31:51.988131 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a2bfe0-d471-40b8-b2d8-fe40303490bc None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:52.068796 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48a8523-831c-4cea-8b62-40c95b710bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:52.069911 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48a8523-831c-4cea-8b62-40c95b710bdc None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:52.070368 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48a8523-831c-4cea-8b62-40c95b710bdc None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:52.070827 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48a8523-831c-4cea-8b62-40c95b710bdc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:52.073316 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:52.084492 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:52.097629 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48a8523-831c-4cea-8b62-40c95b710bdc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:52.098799 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a2bfe0-d471-40b8-b2d8-fe40303490bc None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:52.099257 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48a8523-831c-4cea-8b62-40c95b710bdc None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:52.135858 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690d3e16-0a47-445b-a2e3-106f161a8acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:52.136549 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d3e16-0a47-445b-a2e3-106f161a8acc None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:52.136906 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d3e16-0a47-445b-a2e3-106f161a8acc None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:52.137285 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d3e16-0a47-445b-a2e3-106f161a8acc None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:52.138484 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690d3e16-0a47-445b-a2e3-106f161a8acc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:52.138916 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690d3e16-0a47-445b-a2e3-106f161a8acc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:52.140745 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690d3e16-0a47-445b-a2e3-106f161a8acc None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:52.144894 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-f48a8523-831c-4cea-8b62-40c95b710bdc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:52.144894 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:52.144894 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:52.144894 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:52.144894 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:52.144894 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:52.144894 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:52.144894 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:52.144894 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:52.144894 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:52.144894 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:52.144894 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:52.144894 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:31:52.144894 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:31:52.144894 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:31:52.145811 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:31:52.145811 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:52.145811 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:52.145811 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:52.145811 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:52.145811 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:52.145811 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:52.145811 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:52.145811 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:52.145811 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:52.145811 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:52.145811 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:52.145811 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:52.145811 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:52.145811 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:52.145811 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:52.145811 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:52.146757 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:31:52.146757 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:31:52.146757 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:52.146757 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:52.146757 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 354/710] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:31:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:52.159122 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 357/711] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jun 15 14:31:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:31:52.160375 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48a8523-831c-4cea-8b62-40c95b710bdc None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:52.170720 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690d3e16-0a47-445b-a2e3-106f161a8acc None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:52.339014 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48a8523-831c-4cea-8b62-40c95b710bdc None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:52.369549 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690d3e16-0a47-445b-a2e3-106f161a8acc None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:52.415542 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a003c82e-46b7-4005-831c-e34a7fad2384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:52.416264 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a003c82e-46b7-4005-831c-e34a7fad2384 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:52.416645 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a003c82e-46b7-4005-831c-e34a7fad2384 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:52.416923 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a003c82e-46b7-4005-831c-e34a7fad2384 None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:52.417644 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a003c82e-46b7-4005-831c-e34a7fad2384 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:52.418462 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a003c82e-46b7-4005-831c-e34a7fad2384 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:52.420056 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a003c82e-46b7-4005-831c-e34a7fad2384 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:52.444298 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca93a42-f0f4-4414-bb3e-f07c7a44a402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:52.445280 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca93a42-f0f4-4414-bb3e-f07c7a44a402 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:52.445782 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca93a42-f0f4-4414-bb3e-f07c7a44a402 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:52.446256 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca93a42-f0f4-4414-bb3e-f07c7a44a402 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:52.446464 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 355/712] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:31:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:52.447577 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca93a42-f0f4-4414-bb3e-f07c7a44a402 None admin] RBAC: Adding query filter params (role.id=2bd1979f53bb4853a50dd5bee6aa0782, scope.project.id=3094ff8966f149bda784da6a51b1358f, user.id=0927c05c340c48e48b18754888aa9c35) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:52.448662 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a003c82e-46b7-4005-831c-e34a7fad2384 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:52.450101 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca93a42-f0f4-4414-bb3e-f07c7a44a402 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:52.453593 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca93a42-f0f4-4414-bb3e-f07c7a44a402 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:52.468029 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 358/713] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jun 15 14:31:52 2022] GET /identity/v3/role_assignments?user.id=0927c05c340c48e48b18754888aa9c35&scope.project.id=3094ff8966f149bda784da6a51b1358f&role.id=2bd1979f53bb4853a50dd5bee6aa0782 => generated 259 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:52.527251 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:52.626641 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a003c82e-46b7-4005-831c-e34a7fad2384 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:52.673687 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834d742e-1404-43df-90a8-901f680d7f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:52.674365 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834d742e-1404-43df-90a8-901f680d7f80 None admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:52.674647 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834d742e-1404-43df-90a8-901f680d7f80 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:52.674923 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834d742e-1404-43df-90a8-901f680d7f80 None admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:52.675630 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834d742e-1404-43df-90a8-901f680d7f80 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:52.677094 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834d742e-1404-43df-90a8-901f680d7f80 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:52.776687 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 356/714] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Jun 15 14:31:52 2022] POST /identity/v3/users => generated 254 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:31:52.867631 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb67f3e-1f15-46aa-bb3a-9506bfb27426 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:52.867946 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb67f3e-1f15-46aa-bb3a-9506bfb27426 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:52.868210 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb67f3e-1f15-46aa-bb3a-9506bfb27426 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:52.869890 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 359/715] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:31:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:52.875459 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b999093b-2916-4198-b658-3c4dfd7241be None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:52.875755 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b999093b-2916-4198-b658-3c4dfd7241be None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:52.876026 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b999093b-2916-4198-b658-3c4dfd7241be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:52.898501 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-b999093b-2916-4198-b658-3c4dfd7241be None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:52.900034 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-b999093b-2916-4198-b658-3c4dfd7241be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:52.989109 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 357/716] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:31:53.012833 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb67f3e-1f15-46aa-bb3a-9506bfb27426 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:53.035929 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:53.154064 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb67f3e-1f15-46aa-bb3a-9506bfb27426 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:53.195492 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0676bf04-1957-47fd-aadc-ee3519f8300e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:53.196345 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0676bf04-1957-47fd-aadc-ee3519f8300e None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:53.196632 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0676bf04-1957-47fd-aadc-ee3519f8300e None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:53.196909 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0676bf04-1957-47fd-aadc-ee3519f8300e None admin] PATH_INFO: `/v3/services/network` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:53.197568 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0676bf04-1957-47fd-aadc-ee3519f8300e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:53.198947 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0676bf04-1957-47fd-aadc-ee3519f8300e None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:53.211676 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-0676bf04-1957-47fd-aadc-ee3519f8300e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 14:31:53.211676 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:53.211676 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:31:53.211676 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:31:53.211676 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:31:53.211676 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:31:53.211676 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:31:53.211676 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:31:53.211676 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 14:31:53.211676 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:53.211676 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:31:53.211676 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:53.211676 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:53.211676 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:53.211676 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:53.211676 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:53.211676 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:53.212707 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:53.212707 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:53.212707 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:53.212707 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:53.212707 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:53.212707 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:53.212707 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:31:53.212707 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:31:53.212707 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:31:53.212707 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:31:53.212707 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:53.212707 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:53.212707 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:53.212707 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:53.212707 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:53.212707 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:53.213674 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:53.213674 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:53.213674 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:53.213674 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:53.213674 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:53.213674 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:53.213674 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:53.213674 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:53.213674 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:53.213674 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:53.213674 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:31:53.213674 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:31:53.213674 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 14:31:53.213674 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:53.213674 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 360/717] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jun 15 14:31:53 2022] GET /identity/v3/services/network => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:53.221296 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b999093b-2916-4198-b658-3c4dfd7241be None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:53.333411 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:53.470781 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b999093b-2916-4198-b658-3c4dfd7241be None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:53.527984 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a170a8-4eab-40e1-9f39-97e24d609b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:53.530228 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a170a8-4eab-40e1-9f39-97e24d609b9c None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:53.531384 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a170a8-4eab-40e1-9f39-97e24d609b9c None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:53.532159 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a170a8-4eab-40e1-9f39-97e24d609b9c None admin] PATH_INFO: `/v3/services` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:53.533375 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a170a8-4eab-40e1-9f39-97e24d609b9c None admin] RBAC: Adding query filter params (name=network) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:53.544586 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a170a8-4eab-40e1-9f39-97e24d609b9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:53.561850 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a170a8-4eab-40e1-9f39-97e24d609b9c None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:53.575485 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 358/718] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Jun 15 14:31:53 2022] GET /identity/v3/services?name=network => generated 124 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:53.584796 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0676bf04-1957-47fd-aadc-ee3519f8300e None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:53.747953 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:53.778736 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0676bf04-1957-47fd-aadc-ee3519f8300e None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:53.805099 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca69c159-4415-44c9-a8dc-63f95ef4f690 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:53.805711 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca69c159-4415-44c9-a8dc-63f95ef4f690 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:53.806189 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca69c159-4415-44c9-a8dc-63f95ef4f690 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:53.807740 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 359/719] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:31:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:53.814592 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5691d8c1-b660-4684-b0a8-e7b46feac119 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:53.814754 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5691d8c1-b660-4684-b0a8-e7b46feac119 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:53.815050 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5691d8c1-b660-4684-b0a8-e7b46feac119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:53.833812 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2733d766-f106-4808-a537-8cc465e3a15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:53.834897 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2733d766-f106-4808-a537-8cc465e3a15c None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:53.835421 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2733d766-f106-4808-a537-8cc465e3a15c None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:53.835918 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2733d766-f106-4808-a537-8cc465e3a15c None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:53.837153 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2733d766-f106-4808-a537-8cc465e3a15c None admin] RBAC: Adding query filter params (type=network) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:53.839009 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2733d766-f106-4808-a537-8cc465e3a15c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:53.843025 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2733d766-f106-4808-a537-8cc465e3a15c None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:53.850356 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 361/720] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Jun 15 14:31:53 2022] GET /identity/v3/services?type=network => generated 124 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:53.863192 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-5691d8c1-b660-4684-b0a8-e7b46feac119 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:53.864648 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-5691d8c1-b660-4684-b0a8-e7b46feac119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:54.050795 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 360/721] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:31:54.085133 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef625eaa-26bc-4ee1-90d9-4a8f50fc4383 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:54.085447 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef625eaa-26bc-4ee1-90d9-4a8f50fc4383 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:54.085551 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5691d8c1-b660-4684-b0a8-e7b46feac119 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:54.085646 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef625eaa-26bc-4ee1-90d9-4a8f50fc4383 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:54.087290 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 362/722] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:31:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:54.093200 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd102fa-b407-4070-9bfe-7ed1140f994c None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:54.093488 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd102fa-b407-4070-9bfe-7ed1140f994c None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:54.093775 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd102fa-b407-4070-9bfe-7ed1140f994c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:54.121116 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:54.131650 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-dbd102fa-b407-4070-9bfe-7ed1140f994c None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:54.132906 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-dbd102fa-b407-4070-9bfe-7ed1140f994c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:54.267582 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5691d8c1-b660-4684-b0a8-e7b46feac119 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:54.294359 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 363/723] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:31:54.319565 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ead9682-6083-42c7-b99c-984368128952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:54.320404 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead9682-6083-42c7-b99c-984368128952 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:54.320692 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead9682-6083-42c7-b99c-984368128952 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:54.320974 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead9682-6083-42c7-b99c-984368128952 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:54.337273 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ead9682-6083-42c7-b99c-984368128952 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:31:54.345464 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-9ead9682-6083-42c7-b99c-984368128952 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:31:54.345464 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:54.345464 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:54.345464 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:54.345464 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:54.345464 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:54.345464 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:54.345464 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:54.345464 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:54.345464 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:54.345464 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:54.345464 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:54.345464 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:31:54.345464 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:31:54.345464 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:31:54.345464 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:31:54.346374 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:31:54.346374 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:31:54.346374 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:31:54.346374 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:31:54.346374 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:54.346374 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:54.346374 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:54.346374 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:54.346374 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:54.346374 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:54.346374 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:54.346374 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:54.346374 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:54.346374 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:54.346374 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:54.346374 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:54.347824 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:54.347824 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:54.347824 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:54.347824 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:54.347824 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:31:54.347824 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:31:54.347824 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:31:54.347824 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:31:54.347824 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:31:54.347824 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:31:54.347824 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:31:54.347824 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:54.347824 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 361/724] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jun 15 14:31:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:54.352527 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd102fa-b407-4070-9bfe-7ed1140f994c None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:54.356121 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ead9682-6083-42c7-b99c-984368128952 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:54.448911 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:54.465790 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ead9682-6083-42c7-b99c-984368128952 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:54.515311 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d1f60e-7f3f-466d-8be4-759fb604eb7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:54.515997 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d1f60e-7f3f-466d-8be4-759fb604eb7b None admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:54.516320 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d1f60e-7f3f-466d-8be4-759fb604eb7b None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:54.516601 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d1f60e-7f3f-466d-8be4-759fb604eb7b None admin] PATH_INFO: `/v3/services` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:54.517330 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d1f60e-7f3f-466d-8be4-759fb604eb7b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:54.518718 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d1f60e-7f3f-466d-8be4-759fb604eb7b None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:54.525839 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:54.560758 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd102fa-b407-4070-9bfe-7ed1140f994c None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:54.607242 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66217a53-9598-4527-a7da-ac551769251c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:54.607852 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66217a53-9598-4527-a7da-ac551769251c None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:54.608249 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66217a53-9598-4527-a7da-ac551769251c None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:54.608536 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66217a53-9598-4527-a7da-ac551769251c None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:54.609343 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66217a53-9598-4527-a7da-ac551769251c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:54.610592 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66217a53-9598-4527-a7da-ac551769251c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:54.611157 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66217a53-9598-4527-a7da-ac551769251c None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:54.620928 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 364/725] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jun 15 14:31:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:54.629124 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66217a53-9598-4527-a7da-ac551769251c None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:54.644104 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 362/726] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Jun 15 14:31:54 2022] POST /identity/v3/services => generated 251 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:31:54.770484 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66217a53-9598-4527-a7da-ac551769251c None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:54.804691 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:54.828779 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01ffc4c-b5f8-453c-aa3d-debabf43b6a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:54.829594 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ffc4c-b5f8-453c-aa3d-debabf43b6a8 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:54.829877 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ffc4c-b5f8-453c-aa3d-debabf43b6a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:54.830154 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ffc4c-b5f8-453c-aa3d-debabf43b6a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:54.842783 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01ffc4c-b5f8-453c-aa3d-debabf43b6a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:54.844347 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01ffc4c-b5f8-453c-aa3d-debabf43b6a8 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:54.862840 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-f01ffc4c-b5f8-453c-aa3d-debabf43b6a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:54.862840 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:54.862840 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:54.862840 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:54.862840 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:54.862840 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:54.862840 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:54.862840 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:54.862840 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:54.862840 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:54.862840 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:54.862840 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:54.862840 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:31:54.862840 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:31:54.862840 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:31:54.863748 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:31:54.863748 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:54.863748 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:54.863748 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:54.863748 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:54.863748 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:54.863748 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:54.863748 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:54.863748 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:54.863748 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:54.863748 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:54.863748 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:54.863748 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:54.863748 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:54.863748 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:54.863748 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:54.863748 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:54.865679 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:31:54.865679 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:31:54.865679 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:54.865679 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:54.865679 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 365/727] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:31:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:54.870010 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d1f60e-7f3f-466d-8be4-759fb604eb7b None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:54.981871 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d1f60e-7f3f-466d-8be4-759fb604eb7b None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:55.019349 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dfe7c9-9355-4476-9ba6-7b2dc835bf2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:55.020051 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dfe7c9-9355-4476-9ba6-7b2dc835bf2e None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:55.020367 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dfe7c9-9355-4476-9ba6-7b2dc835bf2e None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:55.020653 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dfe7c9-9355-4476-9ba6-7b2dc835bf2e None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:55.021369 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dfe7c9-9355-4476-9ba6-7b2dc835bf2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:55.021679 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dfe7c9-9355-4476-9ba6-7b2dc835bf2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:55.023045 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dfe7c9-9355-4476-9ba6-7b2dc835bf2e None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:55.036507 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 363/728] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:31:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:55.042831 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01ffc4c-b5f8-453c-aa3d-debabf43b6a8 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:55.163014 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01ffc4c-b5f8-453c-aa3d-debabf43b6a8 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:55.206597 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31171ec-fc81-45b1-8cc7-80a06ec71b27 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:55.207239 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9bf9f5-291c-4283-beb3-9b3c5f44f03b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:55.207478 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9bf9f5-291c-4283-beb3-9b3c5f44f03b None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:55.207768 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9bf9f5-291c-4283-beb3-9b3c5f44f03b None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:55.208042 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9bf9f5-291c-4283-beb3-9b3c5f44f03b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:55.208411 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31171ec-fc81-45b1-8cc7-80a06ec71b27 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:55.208906 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31171ec-fc81-45b1-8cc7-80a06ec71b27 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:55.214434 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 364/729] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:31:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:55.217075 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663e6cf6-ad12-40d9-a16d-ef3bd6fdcf1d None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:55.217604 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663e6cf6-ad12-40d9-a16d-ef3bd6fdcf1d None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:55.218084 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663e6cf6-ad12-40d9-a16d-ef3bd6fdcf1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:55.225243 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9bf9f5-291c-4283-beb3-9b3c5f44f03b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:55.230350 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:55.233959 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9bf9f5-291c-4283-beb3-9b3c5f44f03b None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:55.240392 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-663e6cf6-ad12-40d9-a16d-ef3bd6fdcf1d None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:55.241582 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-663e6cf6-ad12-40d9-a16d-ef3bd6fdcf1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:55.256382 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-9e9bf9f5-291c-4283-beb3-9b3c5f44f03b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 15 14:31:55.256382 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:55.256382 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:55.256382 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:55.256382 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:55.256382 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:55.256382 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:55.256382 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:55.256382 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:55.256382 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:55.256382 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:55.256382 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:55.256382 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:31:55.256382 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:31:55.256382 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:31:55.256382 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:31:55.257401 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:55.257401 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:55.257401 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:31:55.257401 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:31:55.257401 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:31:55.257401 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:31:55.257401 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:55.257401 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:55.257401 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:55.257401 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:55.257401 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:55.257401 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:55.257401 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:55.257401 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:55.257401 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:55.257401 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:55.257401 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:55.257401 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:55.258348 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:55.258348 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:55.258348 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:31:55.258348 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:31:55.258348 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:31:55.258348 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:31:55.258348 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:31:55.258348 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:31:55.258348 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 15 14:31:55.258348 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:55.267594 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 366/730] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jun 15 14:31:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:31:55.270279 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9bf9f5-291c-4283-beb3-9b3c5f44f03b None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:55.362789 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:55.369023 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 365/731] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:55 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:31:55.409794 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663e6cf6-ad12-40d9-a16d-ef3bd6fdcf1d None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:55.492881 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9bf9f5-291c-4283-beb3-9b3c5f44f03b None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:55.573449 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61290e8-fc31-4f37-8ac5-47664994460d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:55.574151 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61290e8-fc31-4f37-8ac5-47664994460d None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:55.574622 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61290e8-fc31-4f37-8ac5-47664994460d None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:55.575128 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61290e8-fc31-4f37-8ac5-47664994460d None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:55.576048 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61290e8-fc31-4f37-8ac5-47664994460d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:55.576392 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61290e8-fc31-4f37-8ac5-47664994460d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:55.577830 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61290e8-fc31-4f37-8ac5-47664994460d None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:55.602608 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 367/732] 10.222.0.42 () {56 vars in 1135 bytes} [Wed Jun 15 14:31:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:55.603983 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61290e8-fc31-4f37-8ac5-47664994460d None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:55.606708 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663e6cf6-ad12-40d9-a16d-ef3bd6fdcf1d None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:55.650119 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2e348b-bd61-4859-8e94-6f27730ae8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:55.650908 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2e348b-bd61-4859-8e94-6f27730ae8ff None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:55.651184 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2e348b-bd61-4859-8e94-6f27730ae8ff None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:55.651452 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2e348b-bd61-4859-8e94-6f27730ae8ff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:55.656377 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:55.672071 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2e348b-bd61-4859-8e94-6f27730ae8ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:31:55.684456 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-ac2e348b-bd61-4859-8e94-6f27730ae8ff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:31:55.684456 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:55.684456 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:55.684456 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:55.684456 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:55.684456 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:55.684456 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:55.684456 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:55.684456 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:55.684456 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:55.684456 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:55.684456 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:55.684456 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:31:55.684456 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:31:55.684456 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:31:55.684456 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:31:55.685415 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:31:55.685415 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:31:55.685415 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:31:55.685415 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:31:55.685415 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:55.685415 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:55.685415 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:55.685415 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:55.685415 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:55.685415 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:55.685415 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:55.685415 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:55.685415 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:55.685415 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:55.685415 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:55.685415 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:55.686251 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:55.686251 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:55.686251 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:55.686251 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:55.686251 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:31:55.686251 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:31:55.686251 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:31:55.686251 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:31:55.686251 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:31:55.686251 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:31:55.686251 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:31:55.686251 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:55.688108 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 366/733] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jun 15 14:31:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:31:55.692672 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2e348b-bd61-4859-8e94-6f27730ae8ff None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:55.776409 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:55.782081 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61290e8-fc31-4f37-8ac5-47664994460d None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:55.823951 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e77f21-37a7-41e7-9b10-b9021c841d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:55.825258 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e77f21-37a7-41e7-9b10-b9021c841d37 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:55.825740 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e77f21-37a7-41e7-9b10-b9021c841d37 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:55.826488 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e77f21-37a7-41e7-9b10-b9021c841d37 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:55.880793 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2e348b-bd61-4859-8e94-6f27730ae8ff None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:55.926026 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c38054-666c-4695-95e0-357ab8f45ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:55.927157 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c38054-666c-4695-95e0-357ab8f45ee8 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:55.928115 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c38054-666c-4695-95e0-357ab8f45ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:55.928542 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c38054-666c-4695-95e0-357ab8f45ee8 None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:55.929745 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c38054-666c-4695-95e0-357ab8f45ee8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:55.929881 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e77f21-37a7-41e7-9b10-b9021c841d37 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:55.930339 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c38054-666c-4695-95e0-357ab8f45ee8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:55.931233 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e77f21-37a7-41e7-9b10-b9021c841d37 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:55.932029 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c38054-666c-4695-95e0-357ab8f45ee8 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:55.943604 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 367/734] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jun 15 14:31:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:55.945800 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceff0e35-963c-4a14-a8a4-826877d81dce None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:55.946235 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceff0e35-963c-4a14-a8a4-826877d81dce None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:55.946899 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceff0e35-963c-4a14-a8a4-826877d81dce None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:55.949074 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 368/735] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:31:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:55.949986 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceff0e35-963c-4a14-a8a4-826877d81dce None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:56.024384 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:56.049717 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:56.054585 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 368/736] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jun 15 14:31:55 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 15 14:31:56.056186 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee08b994-82c7-42e4-b78b-786869709b09 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:56.056651 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee08b994-82c7-42e4-b78b-786869709b09 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:56.057077 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee08b994-82c7-42e4-b78b-786869709b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:56.070140 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceff0e35-963c-4a14-a8a4-826877d81dce None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:56.076080 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-ee08b994-82c7-42e4-b78b-786869709b09 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:56.077592 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-ee08b994-82c7-42e4-b78b-786869709b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:56.108477 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e7b7b1-6542-463c-8bbc-58ed097ea564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:56.109123 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e7b7b1-6542-463c-8bbc-58ed097ea564 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:56.109409 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e7b7b1-6542-463c-8bbc-58ed097ea564 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:56.109756 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e7b7b1-6542-463c-8bbc-58ed097ea564 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:56.120798 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e7b7b1-6542-463c-8bbc-58ed097ea564 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:56.122375 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e7b7b1-6542-463c-8bbc-58ed097ea564 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:56.141568 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-56e7b7b1-6542-463c-8bbc-58ed097ea564 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:56.141568 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:56.141568 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:56.141568 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:56.141568 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:56.141568 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:56.141568 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:56.141568 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:56.141568 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:56.141568 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:56.141568 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:56.141568 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:56.141568 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:31:56.141568 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:31:56.141568 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:31:56.142777 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:31:56.142777 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:56.142777 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:56.142777 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:56.142777 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:56.142777 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:56.142777 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:56.142777 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:56.142777 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:56.142777 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:56.142777 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:56.142777 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:56.142777 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:56.142777 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:56.142777 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:56.142777 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:56.142777 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:56.143854 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:31:56.143854 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:31:56.143854 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:56.143854 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:56.143854 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 369/737] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:31:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:56.144168 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e7b7b1-6542-463c-8bbc-58ed097ea564 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:56.158498 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 369/738] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:56 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:31:56.159614 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee08b994-82c7-42e4-b78b-786869709b09 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:56.269263 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee08b994-82c7-42e4-b78b-786869709b09 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:56.279520 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e7b7b1-6542-463c-8bbc-58ed097ea564 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:56.290908 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:56.300926 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:56.309958 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cea4a2-5ce8-4567-be73-00ced0d5562f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:56.310834 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cea4a2-5ce8-4567-be73-00ced0d5562f None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:56.311263 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cea4a2-5ce8-4567-be73-00ced0d5562f None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:56.311711 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cea4a2-5ce8-4567-be73-00ced0d5562f None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:56.312615 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cea4a2-5ce8-4567-be73-00ced0d5562f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:56.313067 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cea4a2-5ce8-4567-be73-00ced0d5562f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:56.314791 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cea4a2-5ce8-4567-be73-00ced0d5562f None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:56.318078 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0e048b-2d72-4a76-9018-7986d526e67f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:56.318500 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0e048b-2d72-4a76-9018-7986d526e67f None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:56.318787 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0e048b-2d72-4a76-9018-7986d526e67f None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:56.319066 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0e048b-2d72-4a76-9018-7986d526e67f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:56.328536 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 370/739] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:31:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:56.330598 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cea4a2-5ce8-4567-be73-00ced0d5562f None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:56.331052 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0e048b-2d72-4a76-9018-7986d526e67f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:56.332930 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0e048b-2d72-4a76-9018-7986d526e67f None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:56.351522 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-af0e048b-2d72-4a76-9018-7986d526e67f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:56.351522 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:56.351522 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:56.351522 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:56.351522 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:56.351522 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:56.351522 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:56.351522 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:56.351522 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:56.351522 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:56.351522 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:56.351522 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:56.351522 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:31:56.351522 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:31:56.351522 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:31:56.352753 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:31:56.352753 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:56.352753 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:56.352753 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:56.352753 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:56.352753 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:56.352753 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:56.352753 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:56.352753 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:56.352753 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:56.352753 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:56.352753 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:56.352753 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:56.352753 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:56.352753 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:56.352753 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:56.352753 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:56.353815 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:31:56.353815 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:31:56.353815 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:56.353815 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:56.353815 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 370/740] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:31:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:56.353815 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0e048b-2d72-4a76-9018-7986d526e67f None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:56.446949 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cea4a2-5ce8-4567-be73-00ced0d5562f None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:56.470758 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0e048b-2d72-4a76-9018-7986d526e67f None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:56.485134 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd77535a-f540-464d-949e-0ab0cd8e6c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:56.485936 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd77535a-f540-464d-949e-0ab0cd8e6c1d None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:56.486211 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd77535a-f540-464d-949e-0ab0cd8e6c1d None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:56.486570 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd77535a-f540-464d-949e-0ab0cd8e6c1d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:56.497470 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd77535a-f540-464d-949e-0ab0cd8e6c1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:31:56.505148 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-dd77535a-f540-464d-949e-0ab0cd8e6c1d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:31:56.505148 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:56.505148 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:56.505148 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:56.505148 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:56.505148 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:56.505148 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:56.505148 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:56.505148 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:56.505148 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:56.505148 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:56.505148 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:56.505148 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:31:56.505148 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:31:56.505148 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:31:56.505148 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:31:56.506108 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:31:56.506108 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:31:56.506108 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:31:56.506108 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:31:56.506108 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:56.506108 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:56.506108 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:56.506108 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:56.506108 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:56.506108 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:56.506108 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:56.506108 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:56.506108 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:56.506108 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:56.506108 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:56.506108 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:56.507641 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:56.507641 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:56.507641 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:56.507641 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:56.507641 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:31:56.507641 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:31:56.507641 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:31:56.507641 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:31:56.507641 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:31:56.507641 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:31:56.507641 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:31:56.507641 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:56.507641 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 371/741] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Jun 15 14:31:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:56.507641 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd77535a-f540-464d-949e-0ab0cd8e6c1d None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:56.514828 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f547764-f6a1-4c6e-b7f9-e55b990566b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:56.515630 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f547764-f6a1-4c6e-b7f9-e55b990566b6 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:56.515913 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f547764-f6a1-4c6e-b7f9-e55b990566b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:56.516230 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f547764-f6a1-4c6e-b7f9-e55b990566b6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:56.524366 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f547764-f6a1-4c6e-b7f9-e55b990566b6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:56.525874 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f547764-f6a1-4c6e-b7f9-e55b990566b6 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:56.542776 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-9f547764-f6a1-4c6e-b7f9-e55b990566b6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:31:56.542776 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:56.542776 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:56.542776 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:56.542776 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:56.542776 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:56.542776 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:56.542776 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:56.542776 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:56.542776 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:56.542776 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:56.542776 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:56.542776 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:31:56.542776 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:31:56.542776 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:31:56.542776 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:31:56.543773 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:56.543773 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:56.543773 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:31:56.543773 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:31:56.543773 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:31:56.543773 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:31:56.543773 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:56.543773 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:56.543773 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:56.543773 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:56.543773 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:56.543773 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:56.543773 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:56.543773 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:56.543773 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:56.543773 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:56.543773 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:56.543773 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:56.544773 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:56.544773 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:56.544773 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:31:56.544773 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:31:56.544773 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:31:56.544773 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:31:56.544773 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:31:56.544773 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:31:56.544773 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:31:56.544773 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:56.548819 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 371/742] 10.222.0.42 () {56 vars in 1093 bytes} [Wed Jun 15 14:31:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:56.553790 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b0aee1-a9bc-41dd-9b10-25451a705e52 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:56.554251 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b0aee1-a9bc-41dd-9b10-25451a705e52 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:56.554541 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b0aee1-a9bc-41dd-9b10-25451a705e52 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:56.561460 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 372/743] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:31:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:56.561460 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b0aee1-a9bc-41dd-9b10-25451a705e52 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:56.562393 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:56.591452 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:56.615254 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd77535a-f540-464d-949e-0ab0cd8e6c1d None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:56.653701 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea0964d-42de-46d6-ba5b-4f39e08b8cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:56.654362 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0964d-42de-46d6-ba5b-4f39e08b8cd2 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:56.654668 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0964d-42de-46d6-ba5b-4f39e08b8cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:56.654915 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0964d-42de-46d6-ba5b-4f39e08b8cd2 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:56.655636 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea0964d-42de-46d6-ba5b-4f39e08b8cd2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:56.655938 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea0964d-42de-46d6-ba5b-4f39e08b8cd2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:56.657318 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea0964d-42de-46d6-ba5b-4f39e08b8cd2 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:56.670431 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 372/744] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:31:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:56.675591 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea0964d-42de-46d6-ba5b-4f39e08b8cd2 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:56.676517 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b0aee1-a9bc-41dd-9b10-25451a705e52 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:56.715806 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f841a7e-bef0-45ee-99bb-c7ad6eddc9cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:56.716574 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f841a7e-bef0-45ee-99bb-c7ad6eddc9cb None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:56.716858 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f841a7e-bef0-45ee-99bb-c7ad6eddc9cb None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:56.717133 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f841a7e-bef0-45ee-99bb-c7ad6eddc9cb None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:56.718305 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f841a7e-bef0-45ee-99bb-c7ad6eddc9cb None admin] RBAC: Adding query filter params (name=service) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:56.718615 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f841a7e-bef0-45ee-99bb-c7ad6eddc9cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:56.720354 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f841a7e-bef0-45ee-99bb-c7ad6eddc9cb None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:56.734925 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 373/745] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Jun 15 14:31:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:56.734925 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f3c9b-6546-4503-bb91-d9b6c89454dc None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:56.734925 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f3c9b-6546-4503-bb91-d9b6c89454dc None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:56.734925 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f3c9b-6546-4503-bb91-d9b6c89454dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:56.750652 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-847f3c9b-6546-4503-bb91-d9b6c89454dc None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:56.751831 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-847f3c9b-6546-4503-bb91-d9b6c89454dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:56.790162 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea0964d-42de-46d6-ba5b-4f39e08b8cd2 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:56.802379 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:56.823339 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 374/746] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:56 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:31:56.824804 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847f3c9b-6546-4503-bb91-d9b6c89454dc None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:56.840998 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:56.843197 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97ef8a2-3e72-486c-9eba-8876f54418c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:56.844115 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ef8a2-3e72-486c-9eba-8876f54418c0 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:56.844968 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ef8a2-3e72-486c-9eba-8876f54418c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:56.845715 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ef8a2-3e72-486c-9eba-8876f54418c0 None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:56.846791 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97ef8a2-3e72-486c-9eba-8876f54418c0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:56.847284 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97ef8a2-3e72-486c-9eba-8876f54418c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:56.849517 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97ef8a2-3e72-486c-9eba-8876f54418c0 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:56.868987 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 373/747] 10.222.0.42 () {56 vars in 1133 bytes} [Wed Jun 15 14:31:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:56.870182 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97ef8a2-3e72-486c-9eba-8876f54418c0 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:56.938168 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847f3c9b-6546-4503-bb91-d9b6c89454dc None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:56.975780 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b3b219-d801-4cd4-a42e-217cd63a4abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:56.976597 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b3b219-d801-4cd4-a42e-217cd63a4abd None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:56.976878 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b3b219-d801-4cd4-a42e-217cd63a4abd None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:56.977153 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b3b219-d801-4cd4-a42e-217cd63a4abd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:56.982767 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97ef8a2-3e72-486c-9eba-8876f54418c0 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:56.984683 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b3b219-d801-4cd4-a42e-217cd63a4abd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:56.986135 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b3b219-d801-4cd4-a42e-217cd63a4abd None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:57.000442 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-95b3b219-d801-4cd4-a42e-217cd63a4abd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:31:57.000442 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:57.000442 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:57.000442 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:57.000442 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:57.000442 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:57.000442 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:57.000442 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:57.000442 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:57.000442 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:57.000442 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:57.000442 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:57.000442 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:31:57.000442 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:31:57.000442 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:31:57.001450 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:31:57.001450 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:57.001450 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:57.001450 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:57.001450 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:57.001450 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:57.001450 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:57.001450 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:57.001450 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:57.001450 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:57.001450 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:57.001450 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:57.001450 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:57.001450 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:57.001450 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:57.001450 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:57.001450 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:57.002342 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:31:57.002342 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:31:57.002342 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:31:57.002342 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:31:57.002342 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:31:57.002342 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:31:57.002342 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:31:57.002342 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:57.002342 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 375/748] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jun 15 14:31:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:57.002822 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b3b219-d801-4cd4-a42e-217cd63a4abd None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:57.025114 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e644de0c-df16-4a32-ae0a-981b991db1e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:57.025907 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644de0c-df16-4a32-ae0a-981b991db1e4 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:57.026187 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644de0c-df16-4a32-ae0a-981b991db1e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:57.026467 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644de0c-df16-4a32-ae0a-981b991db1e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:57.037750 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e644de0c-df16-4a32-ae0a-981b991db1e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:57.039261 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e644de0c-df16-4a32-ae0a-981b991db1e4 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:57.057938 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-e644de0c-df16-4a32-ae0a-981b991db1e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:57.057938 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:57.057938 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:57.057938 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:57.057938 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:57.057938 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:57.057938 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:57.057938 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:57.057938 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:57.057938 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:57.057938 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:57.057938 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:57.057938 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:31:57.057938 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:31:57.057938 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:31:57.059489 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:31:57.059489 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:57.059489 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:57.059489 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:57.059489 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:57.059489 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:57.059489 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:57.059489 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:57.059489 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:57.059489 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:57.059489 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:57.059489 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:57.059489 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:57.059489 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:57.059489 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:57.059489 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:57.059489 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:57.060390 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:31:57.060390 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:31:57.060390 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:57.060390 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:57.064395 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 374/749] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:31:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:57.065613 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e644de0c-df16-4a32-ae0a-981b991db1e4 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:57.084314 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:57.085394 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:57.109814 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b3b219-d801-4cd4-a42e-217cd63a4abd None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:57.147916 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e4b3a8-992b-4daf-9539-5b37068d8bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:57.148618 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4b3a8-992b-4daf-9539-5b37068d8bfe None admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:57.149517 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4b3a8-992b-4daf-9539-5b37068d8bfe None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:57.149803 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4b3a8-992b-4daf-9539-5b37068d8bfe None admin] PATH_INFO: `/v3/services` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:57.150842 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e4b3a8-992b-4daf-9539-5b37068d8bfe None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:57.153103 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e4b3a8-992b-4daf-9539-5b37068d8bfe None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:57.172605 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e644de0c-df16-4a32-ae0a-981b991db1e4 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:57.204411 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 376/750] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Jun 15 14:31:57 2022] POST /identity/v3/services => generated 240 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:31:57.205387 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e4b3a8-992b-4daf-9539-5b37068d8bfe None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:57.225235 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cc5be3-30e8-4359-85a6-38e1e878412c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:57.226085 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cc5be3-30e8-4359-85a6-38e1e878412c None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:57.226368 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cc5be3-30e8-4359-85a6-38e1e878412c None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:57.226647 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cc5be3-30e8-4359-85a6-38e1e878412c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:57.240810 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cc5be3-30e8-4359-85a6-38e1e878412c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:57.242326 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cc5be3-30e8-4359-85a6-38e1e878412c None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:57.262711 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-34cc5be3-30e8-4359-85a6-38e1e878412c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:57.262711 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:57.262711 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:57.262711 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:57.262711 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:57.262711 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:57.262711 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:57.262711 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:57.262711 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:57.262711 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:57.262711 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:57.262711 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:57.262711 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:31:57.262711 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:31:57.262711 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:31:57.263717 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:31:57.263717 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:57.263717 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:57.263717 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:57.263717 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:57.263717 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:57.263717 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:57.263717 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:57.263717 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:57.263717 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:57.263717 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:57.263717 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:57.263717 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:57.263717 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:57.263717 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:57.263717 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:57.263717 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:57.269669 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:31:57.269669 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:31:57.269669 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:57.269669 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:57.269669 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 375/751] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:31:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:57.269669 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cc5be3-30e8-4359-85a6-38e1e878412c None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:57.341883 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:57.350838 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e4b3a8-992b-4daf-9539-5b37068d8bfe None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:57.380056 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cc5be3-30e8-4359-85a6-38e1e878412c None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:57.393192 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed88253-313d-4e82-b167-777cc057141e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:57.393858 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed88253-313d-4e82-b167-777cc057141e None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:57.394140 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed88253-313d-4e82-b167-777cc057141e None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:57.394416 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed88253-313d-4e82-b167-777cc057141e None admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:57.395198 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed88253-313d-4e82-b167-777cc057141e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:57.395753 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed88253-313d-4e82-b167-777cc057141e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:57.396952 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed88253-313d-4e82-b167-777cc057141e None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:57.398633 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:57.411749 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 377/752] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jun 15 14:31:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:57.412937 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed88253-313d-4e82-b167-777cc057141e None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:57.423871 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaca8cc3-384c-455c-9d9d-d17d2daef34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:57.424585 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca8cc3-384c-455c-9d9d-d17d2daef34c None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:57.424873 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca8cc3-384c-455c-9d9d-d17d2daef34c None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:57.425150 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca8cc3-384c-455c-9d9d-d17d2daef34c None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:57.426072 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaca8cc3-384c-455c-9d9d-d17d2daef34c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:57.426387 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaca8cc3-384c-455c-9d9d-d17d2daef34c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:57.427756 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaca8cc3-384c-455c-9d9d-d17d2daef34c None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:57.440896 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 376/753] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:31:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:57.444151 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844dce6d-f302-49c7-9350-3a7ddcc6d7a4 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:57.444151 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844dce6d-f302-49c7-9350-3a7ddcc6d7a4 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:57.444151 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844dce6d-f302-49c7-9350-3a7ddcc6d7a4 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:57.444151 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 377/754] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:31:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:57.444905 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844dce6d-f302-49c7-9350-3a7ddcc6d7a4 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:57.532969 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed88253-313d-4e82-b167-777cc057141e None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:57.559768 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844dce6d-f302-49c7-9350-3a7ddcc6d7a4 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:57.576594 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c232f22d-abf8-445f-9fa5-5d86b580480d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:57.577610 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c232f22d-abf8-445f-9fa5-5d86b580480d None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:57.578161 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c232f22d-abf8-445f-9fa5-5d86b580480d None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:57.578445 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c232f22d-abf8-445f-9fa5-5d86b580480d None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:57.579169 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c232f22d-abf8-445f-9fa5-5d86b580480d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:57.579478 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c232f22d-abf8-445f-9fa5-5d86b580480d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:57.580868 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c232f22d-abf8-445f-9fa5-5d86b580480d None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:57.586852 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:57.599072 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 378/755] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:31:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:57.599072 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c232f22d-abf8-445f-9fa5-5d86b580480d None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:57.601091 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa6bb5e-4ba8-41bb-acd9-132818e8d579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:57.601831 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa6bb5e-4ba8-41bb-acd9-132818e8d579 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:57.602120 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa6bb5e-4ba8-41bb-acd9-132818e8d579 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:57.602401 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa6bb5e-4ba8-41bb-acd9-132818e8d579 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:57.603266 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa6bb5e-4ba8-41bb-acd9-132818e8d579 None admin] RBAC: Adding query filter params (role.id=2bd1979f53bb4853a50dd5bee6aa0782, scope.project.id=3094ff8966f149bda784da6a51b1358f, user.id=c04dc5832333498e96a7e02bbfb9484c) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:57.603584 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa6bb5e-4ba8-41bb-acd9-132818e8d579 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:57.605045 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa6bb5e-4ba8-41bb-acd9-132818e8d579 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:57.611465 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 378/756] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jun 15 14:31:57 2022] GET /identity/v3/role_assignments?user.id=c04dc5832333498e96a7e02bbfb9484c&scope.project.id=3094ff8966f149bda784da6a51b1358f&role.id=2bd1979f53bb4853a50dd5bee6aa0782 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:57.612709 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caef67c-1953-4169-8a02-bb5816a4b648 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:57.613005 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caef67c-1953-4169-8a02-bb5816a4b648 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:57.613331 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caef67c-1953-4169-8a02-bb5816a4b648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:57.637903 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-7caef67c-1953-4169-8a02-bb5816a4b648 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:57.639341 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-7caef67c-1953-4169-8a02-bb5816a4b648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:57.684504 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:57.724392 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 379/757] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:31:57.725441 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caef67c-1953-4169-8a02-bb5816a4b648 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:57.731095 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c232f22d-abf8-445f-9fa5-5d86b580480d None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:57.772609 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c64657-6802-4187-a4a2-acf92828b37a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:57.774435 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c64657-6802-4187-a4a2-acf92828b37a None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:57.774726 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c64657-6802-4187-a4a2-acf92828b37a None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:57.775061 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c64657-6802-4187-a4a2-acf92828b37a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:57.786832 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c64657-6802-4187-a4a2-acf92828b37a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:57.788350 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c64657-6802-4187-a4a2-acf92828b37a None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:57.810045 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-c3c64657-6802-4187-a4a2-acf92828b37a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 14:31:57.810045 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:57.810045 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:57.810045 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:57.810045 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:57.810045 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:57.810045 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:57.810045 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:57.810045 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:57.810045 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:57.810045 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:57.810045 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:57.810045 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:31:57.810045 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:31:57.810045 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:31:57.810045 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:31:57.811154 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:57.811154 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:57.811154 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:31:57.811154 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:31:57.811154 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:31:57.811154 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:31:57.811154 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:57.811154 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:57.811154 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:57.811154 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:57.811154 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:57.811154 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:57.811154 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:57.811154 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:57.811154 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:57.811154 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:57.811154 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:57.811154 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:57.812931 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:57.812931 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:57.812931 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:31:57.812931 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:31:57.812931 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:31:57.812931 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:31:57.812931 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:31:57.812931 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:31:57.812931 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 14:31:57.812931 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:57.812931 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 379/758] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Jun 15 14:31:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:57.812931 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c64657-6802-4187-a4a2-acf92828b37a None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:57.869887 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caef67c-1953-4169-8a02-bb5816a4b648 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:57.898205 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:57.923668 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed08c971-85f7-46c4-8aee-5ab0b2f3b5ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:57.925064 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed08c971-85f7-46c4-8aee-5ab0b2f3b5ee None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:57.925737 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed08c971-85f7-46c4-8aee-5ab0b2f3b5ee None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:57.926157 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed08c971-85f7-46c4-8aee-5ab0b2f3b5ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:57.936046 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c64657-6802-4187-a4a2-acf92828b37a None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:57.936704 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed08c971-85f7-46c4-8aee-5ab0b2f3b5ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:57.939597 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed08c971-85f7-46c4-8aee-5ab0b2f3b5ee None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:57.951839 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:57.960517 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-ed08c971-85f7-46c4-8aee-5ab0b2f3b5ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:31:57.960517 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:57.960517 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:57.960517 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:57.960517 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:57.960517 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:57.960517 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:57.960517 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:57.960517 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:57.960517 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:57.960517 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:57.960517 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:57.960517 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:31:57.960517 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:31:57.960517 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:31:57.961602 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:31:57.961602 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:57.961602 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:57.961602 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:57.961602 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:57.961602 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:57.961602 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:57.961602 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:57.961602 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:57.961602 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:57.961602 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:57.961602 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:57.961602 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:57.961602 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:57.961602 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:57.961602 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:57.961602 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:57.962678 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:31:57.962678 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:31:57.962678 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:31:57.962678 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:31:57.962678 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:31:57.962678 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:31:57.962678 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:31:57.962678 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:57.965745 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 380/759] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jun 15 14:31:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:57.970156 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed08c971-85f7-46c4-8aee-5ab0b2f3b5ee None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:57.982028 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7166e5b-aaf1-4059-a968-fbe8952a882e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:57.982960 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7166e5b-aaf1-4059-a968-fbe8952a882e None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:57.983318 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7166e5b-aaf1-4059-a968-fbe8952a882e None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:57.983669 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7166e5b-aaf1-4059-a968-fbe8952a882e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:57.984439 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7166e5b-aaf1-4059-a968-fbe8952a882e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:57.986182 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7166e5b-aaf1-4059-a968-fbe8952a882e None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:58.001576 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-e7166e5b-aaf1-4059-a968-fbe8952a882e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 14:31:58.001576 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:58.001576 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:31:58.001576 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:31:58.001576 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:31:58.001576 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:31:58.001576 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:31:58.001576 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:31:58.001576 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 14:31:58.001576 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:58.001576 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:31:58.001576 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:58.001576 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:58.001576 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:58.001576 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:58.001576 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:58.002531 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:58.002531 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:58.002531 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:58.002531 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:58.002531 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:58.002531 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:58.002531 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:58.002531 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:31:58.002531 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:31:58.002531 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:31:58.002531 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:31:58.002531 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:58.002531 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:58.002531 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:58.002531 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:58.002531 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:58.003449 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:58.003449 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:58.003449 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:58.003449 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:58.003449 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:58.003449 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:58.003449 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:58.003449 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:58.003449 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:58.003449 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:58.003449 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:58.003449 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:31:58.003449 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:31:58.003449 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 14:31:58.003449 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:58.003449 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 380/760] 10.222.0.42 () {56 vars in 1129 bytes} [Wed Jun 15 14:31:57 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:58.005008 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7166e5b-aaf1-4059-a968-fbe8952a882e None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:58.117028 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7166e5b-aaf1-4059-a968-fbe8952a882e None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:58.132598 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed08c971-85f7-46c4-8aee-5ab0b2f3b5ee None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:58.151249 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb0fcf2-94bd-4f91-9998-22a6108af68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:58.152735 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb0fcf2-94bd-4f91-9998-22a6108af68a None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:58.153272 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb0fcf2-94bd-4f91-9998-22a6108af68a None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:58.153743 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb0fcf2-94bd-4f91-9998-22a6108af68a None admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:58.154664 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb0fcf2-94bd-4f91-9998-22a6108af68a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:58.155114 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb0fcf2-94bd-4f91-9998-22a6108af68a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:58.156717 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb0fcf2-94bd-4f91-9998-22a6108af68a None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:58.172082 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 381/761] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jun 15 14:31:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:58.173204 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb0fcf2-94bd-4f91-9998-22a6108af68a None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:58.184393 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d41740b-47ce-49bb-8cc1-979e909f1d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:58.185064 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d41740b-47ce-49bb-8cc1-979e909f1d40 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:58.185346 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d41740b-47ce-49bb-8cc1-979e909f1d40 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:58.185617 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d41740b-47ce-49bb-8cc1-979e909f1d40 None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:58.186464 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d41740b-47ce-49bb-8cc1-979e909f1d40 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:58.186771 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d41740b-47ce-49bb-8cc1-979e909f1d40 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:58.192141 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d41740b-47ce-49bb-8cc1-979e909f1d40 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:58.212918 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 381/762] 10.222.0.42 () {56 vars in 1149 bytes} [Wed Jun 15 14:31:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:58.214167 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d41740b-47ce-49bb-8cc1-979e909f1d40 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:58.224193 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:58.270378 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:58.289192 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb0fcf2-94bd-4f91-9998-22a6108af68a None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:58.328465 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eace70-15d3-450b-bf0a-a0e804dd54cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:58.329465 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eace70-15d3-450b-bf0a-a0e804dd54cf None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:58.329677 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eace70-15d3-450b-bf0a-a0e804dd54cf None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:58.330016 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eace70-15d3-450b-bf0a-a0e804dd54cf None admin] PATH_INFO: `/v3/services` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:58.330844 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eace70-15d3-450b-bf0a-a0e804dd54cf None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:58.331211 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eace70-15d3-450b-bf0a-a0e804dd54cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:58.332821 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eace70-15d3-450b-bf0a-a0e804dd54cf None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:58.338300 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 382/763] 10.222.0.42 () {56 vars in 1124 bytes} [Wed Jun 15 14:31:58 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:58.339145 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eace70-15d3-450b-bf0a-a0e804dd54cf None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:58.357403 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d41740b-47ce-49bb-8cc1-979e909f1d40 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:58.410869 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1575ef3-ade6-433c-98ba-234c045fc3d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:58.412151 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1575ef3-ade6-433c-98ba-234c045fc3d7 None admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:58.412585 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1575ef3-ade6-433c-98ba-234c045fc3d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:58.413036 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1575ef3-ade6-433c-98ba-234c045fc3d7 None admin] PATH_INFO: `/v3/projects/3094ff8966f149bda784da6a51b1358f/users/0927c05c340c48e48b18754888aa9c35/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:58.416038 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1575ef3-ade6-433c-98ba-234c045fc3d7 None admin] RBAC: Authorizing `identity:create_grant(project_id=3094ff8966f149bda784da6a51b1358f, user_id=0927c05c340c48e48b18754888aa9c35, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:58.418734 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1575ef3-ade6-433c-98ba-234c045fc3d7 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:58.452572 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eace70-15d3-450b-bf0a-a0e804dd54cf None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:58.454429 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:58.487835 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ed2311-6d1d-461f-9854-982eabba808b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:58.488506 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ed2311-6d1d-461f-9854-982eabba808b None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:58.488880 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ed2311-6d1d-461f-9854-982eabba808b None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:58.489161 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ed2311-6d1d-461f-9854-982eabba808b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:58.513086 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 382/764] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Jun 15 14:31:58 2022] PUT /identity/v3/projects/3094ff8966f149bda784da6a51b1358f/users/0927c05c340c48e48b18754888aa9c35/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:31:58.514983 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1575ef3-ade6-433c-98ba-234c045fc3d7 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:58.628741 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ed2311-6d1d-461f-9854-982eabba808b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:58.630459 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ed2311-6d1d-461f-9854-982eabba808b None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:58.749460 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:58.756965 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 383/765] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jun 15 14:31:58 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:31:58.757556 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1575ef3-ade6-433c-98ba-234c045fc3d7 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:58.761870 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ed2311-6d1d-461f-9854-982eabba808b None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:58.763977 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:58.808249 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb15c23c-4533-4161-9f71-2e344e4567d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:58.808968 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb15c23c-4533-4161-9f71-2e344e4567d0 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:58.809248 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb15c23c-4533-4161-9f71-2e344e4567d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:58.809519 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb15c23c-4533-4161-9f71-2e344e4567d0 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:58.810232 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb15c23c-4533-4161-9f71-2e344e4567d0 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:58.810536 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb15c23c-4533-4161-9f71-2e344e4567d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:58.811890 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb15c23c-4533-4161-9f71-2e344e4567d0 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:58.818770 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 383/766] 10.222.0.42 () {56 vars in 1124 bytes} [Wed Jun 15 14:31:58 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:58.826308 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb15c23c-4533-4161-9f71-2e344e4567d0 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:58.938030 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ed2311-6d1d-461f-9854-982eabba808b None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:58.944387 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb15c23c-4533-4161-9f71-2e344e4567d0 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:58.991234 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb40f84-9011-4751-b765-e45e99675cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:58.991963 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb40f84-9011-4751-b765-e45e99675cdc None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:58.992276 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb40f84-9011-4751-b765-e45e99675cdc None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:58.993528 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb40f84-9011-4751-b765-e45e99675cdc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:58.995455 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb40f84-9011-4751-b765-e45e99675cdc None admin] RBAC: Adding query filter params (interface=public, service_id=4fa0f1c5ff584b03ae0bb9b3c51d8674, region_id=RegionOne) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:58.995768 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb40f84-9011-4751-b765-e45e99675cdc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:58.997804 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb40f84-9011-4751-b765-e45e99675cdc None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:59.012578 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85d36a1-11c5-470f-9389-936e6e098a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:59.013374 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d36a1-11c5-470f-9389-936e6e098a79 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:59.013652 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d36a1-11c5-470f-9389-936e6e098a79 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:59.013923 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d36a1-11c5-470f-9389-936e6e098a79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:59.019142 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 384/767] 10.222.0.42 () {56 vars in 1251 bytes} [Wed Jun 15 14:31:58 2022] GET /identity/v3/endpoints?service_id=4fa0f1c5ff584b03ae0bb9b3c51d8674&interface=public®ion_id=RegionOne => generated 194 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:59.025197 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb40f84-9011-4751-b765-e45e99675cdc None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:59.036882 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85d36a1-11c5-470f-9389-936e6e098a79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:59.038332 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85d36a1-11c5-470f-9389-936e6e098a79 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:59.060110 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-f85d36a1-11c5-470f-9389-936e6e098a79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:59.060110 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:59.060110 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:59.060110 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:59.060110 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:59.060110 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:59.060110 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:59.060110 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:59.060110 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:59.060110 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:59.060110 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:59.060110 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:59.060110 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:31:59.060110 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:31:59.060110 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:31:59.061153 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:31:59.061153 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:59.061153 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:59.061153 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:59.061153 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:59.061153 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:59.061153 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:59.061153 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:59.061153 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:59.061153 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:59.061153 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:59.061153 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:59.061153 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:59.061153 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:59.061153 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:59.061153 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:59.061153 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:59.062062 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:31:59.062062 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:31:59.062062 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:59.062062 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:31:59.062062 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 384/768] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:31:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:59.068795 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85d36a1-11c5-470f-9389-936e6e098a79 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:59.084204 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:59.161575 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:59.196834 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb40f84-9011-4751-b765-e45e99675cdc None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:59.226745 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85d36a1-11c5-470f-9389-936e6e098a79 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:59.239588 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a014c3-577d-4803-9d16-bd51f54b56c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:59.240299 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a014c3-577d-4803-9d16-bd51f54b56c9 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:59.240583 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a014c3-577d-4803-9d16-bd51f54b56c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:59.240858 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a014c3-577d-4803-9d16-bd51f54b56c9 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:59.241541 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a014c3-577d-4803-9d16-bd51f54b56c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:59.243105 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a014c3-577d-4803-9d16-bd51f54b56c9 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:59.249122 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 385/769] 10.222.0.42 () {56 vars in 1087 bytes} [Wed Jun 15 14:31:59 2022] GET /identity/v3/services => generated 772 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:59.268878 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65bca0c-6f56-4ab6-afa7-bfd8a62fbba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:59.269529 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65bca0c-6f56-4ab6-afa7-bfd8a62fbba1 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:59.269812 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65bca0c-6f56-4ab6-afa7-bfd8a62fbba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:59.270083 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65bca0c-6f56-4ab6-afa7-bfd8a62fbba1 None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:59.270893 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65bca0c-6f56-4ab6-afa7-bfd8a62fbba1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:59.271200 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65bca0c-6f56-4ab6-afa7-bfd8a62fbba1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:59.272597 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65bca0c-6f56-4ab6-afa7-bfd8a62fbba1 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:59.287208 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 385/770] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:31:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:59.292648 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a014c3-577d-4803-9d16-bd51f54b56c9 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:59.398620 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:59.409849 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a014c3-577d-4803-9d16-bd51f54b56c9 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:59.447303 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed087e5-b205-4426-a472-55801c39aeb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:59.448134 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed087e5-b205-4426-a472-55801c39aeb4 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:59.448491 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed087e5-b205-4426-a472-55801c39aeb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:59.449763 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed087e5-b205-4426-a472-55801c39aeb4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:59.457775 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed087e5-b205-4426-a472-55801c39aeb4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:59.459267 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed087e5-b205-4426-a472-55801c39aeb4 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:59.475641 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-3ed087e5-b205-4426-a472-55801c39aeb4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:31:59.475641 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:59.475641 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:59.475641 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:59.475641 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:59.475641 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:59.475641 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:59.475641 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:59.475641 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:59.475641 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:59.475641 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:59.475641 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:59.475641 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:31:59.475641 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:31:59.475641 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:31:59.476739 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:31:59.476739 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:59.476739 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:59.476739 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:59.476739 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:59.476739 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:59.476739 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:59.476739 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:59.476739 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:59.476739 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:59.476739 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:59.476739 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:59.476739 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:59.476739 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:59.476739 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:59.476739 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:59.476739 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:59.477631 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:31:59.477631 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:31:59.477631 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:31:59.477631 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:31:59.477631 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:31:59.477631 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:31:59.477631 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:31:59.477631 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:59.477631 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 386/771] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jun 15 14:31:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:59.484568 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65bca0c-6f56-4ab6-afa7-bfd8a62fbba1 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:59.560189 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d436ebc4-ef42-4779-aabc-094cdb55fdd1 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:59.560710 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d436ebc4-ef42-4779-aabc-094cdb55fdd1 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:59.561221 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d436ebc4-ef42-4779-aabc-094cdb55fdd1 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:59.562900 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 387/772] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:31:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:31:59.567815 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a5c66-4034-40a7-946c-efdd031ec230 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:59.568117 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a5c66-4034-40a7-946c-efdd031ec230 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:59.568414 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a5c66-4034-40a7-946c-efdd031ec230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:59.588638 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-ca1a5c66-4034-40a7-946c-efdd031ec230 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:59.590522 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-ca1a5c66-4034-40a7-946c-efdd031ec230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:59.604466 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65bca0c-6f56-4ab6-afa7-bfd8a62fbba1 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:59.620455 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:59.643712 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ca081f-1cbb-4b48-80cc-89b5ca5cb5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:59.644432 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ca081f-1cbb-4b48-80cc-89b5ca5cb5d1 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:59.644805 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ca081f-1cbb-4b48-80cc-89b5ca5cb5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:59.645084 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ca081f-1cbb-4b48-80cc-89b5ca5cb5d1 None admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:59.645980 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ca081f-1cbb-4b48-80cc-89b5ca5cb5d1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:59.646298 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ca081f-1cbb-4b48-80cc-89b5ca5cb5d1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:59.647804 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ca081f-1cbb-4b48-80cc-89b5ca5cb5d1 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:59.663137 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 386/773] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jun 15 14:31:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:59.674053 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ca081f-1cbb-4b48-80cc-89b5ca5cb5d1 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:59.685874 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 388/774] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:31:59.722883 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1a5c66-4034-40a7-946c-efdd031ec230 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:59.820801 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:59.861259 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1a5c66-4034-40a7-946c-efdd031ec230 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:59.877804 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ca081f-1cbb-4b48-80cc-89b5ca5cb5d1 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:59.916970 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0802644-d50a-4327-bd38-675368929793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:59.918274 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0802644-d50a-4327-bd38-675368929793 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:59.918559 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0802644-d50a-4327-bd38-675368929793 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:59.918828 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0802644-d50a-4327-bd38-675368929793 None admin] PATH_INFO: `/v3/services/network` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:59.919478 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0802644-d50a-4327-bd38-675368929793 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:59.920870 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0802644-d50a-4327-bd38-675368929793 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:59.944768 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-c0802644-d50a-4327-bd38-675368929793 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 14:31:59.944768 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:59.944768 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:31:59.944768 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:31:59.944768 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:31:59.944768 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:31:59.944768 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:31:59.944768 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:31:59.944768 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 14:31:59.944768 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:59.944768 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:31:59.944768 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:59.944768 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:59.944768 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:59.944768 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:59.944768 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:59.944768 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:59.946294 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:59.946294 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:59.946294 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:59.946294 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:59.946294 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:59.946294 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:59.946294 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:31:59.946294 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:31:59.946294 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:31:59.946294 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:31:59.946294 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:59.946294 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:59.946294 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:59.946294 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:59.946294 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:59.946294 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:59.947152 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:59.947152 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:59.947152 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:59.947152 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:59.947152 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:59.947152 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:59.947152 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:59.947152 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:59.947152 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:59.947152 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:59.947152 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:31:59.947152 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:31:59.947152 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 14:31:59.947152 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:31:59.947152 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 389/775] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jun 15 14:31:59 2022] GET /identity/v3/services/network => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:31:59.952914 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0802644-d50a-4327-bd38-675368929793 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:59.956519 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5769b3-0797-4486-a1fa-b9188f4134bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:59.957266 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5769b3-0797-4486-a1fa-b9188f4134bd None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:59.957548 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5769b3-0797-4486-a1fa-b9188f4134bd None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:59.957821 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5769b3-0797-4486-a1fa-b9188f4134bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:59.958652 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5769b3-0797-4486-a1fa-b9188f4134bd None admin] RBAC: Adding query filter params (role.id=305a1ca2e6f740eba7dc696849e933eb, scope.project.id=3094ff8966f149bda784da6a51b1358f, user.id=19d82c04f9994654adfb10f4bd0573c2) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:59.958963 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5769b3-0797-4486-a1fa-b9188f4134bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:59.960661 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5769b3-0797-4486-a1fa-b9188f4134bd None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:59.976138 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 387/776] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jun 15 14:31:59 2022] GET /identity/v3/role_assignments?user.id=19d82c04f9994654adfb10f4bd0573c2&scope.project.id=3094ff8966f149bda784da6a51b1358f&role.id=305a1ca2e6f740eba7dc696849e933eb => generated 259 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:00.127356 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0802644-d50a-4327-bd38-675368929793 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:00.177622 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9c270b-59c2-4407-a350-53179cf8e724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:00.178292 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9c270b-59c2-4407-a350-53179cf8e724 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:00.178577 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9c270b-59c2-4407-a350-53179cf8e724 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:00.178865 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9c270b-59c2-4407-a350-53179cf8e724 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:00.179576 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9c270b-59c2-4407-a350-53179cf8e724 None admin] RBAC: Adding query filter params (name=network) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:00.179888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9c270b-59c2-4407-a350-53179cf8e724 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:00.181289 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9c270b-59c2-4407-a350-53179cf8e724 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:00.187991 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a372e-ece6-484e-a75b-d275878b32dc None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:00.188307 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a372e-ece6-484e-a75b-d275878b32dc None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:00.188569 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a372e-ece6-484e-a75b-d275878b32dc None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:00.190191 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 388/777] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:32:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:32:00.193244 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 390/778] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Jun 15 14:31:59 2022] GET /identity/v3/services?name=network => generated 124 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:00.195719 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b46f5f-5a89-4ef3-a467-243f61da2fcf None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:00.196002 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b46f5f-5a89-4ef3-a467-243f61da2fcf None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:00.196305 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b46f5f-5a89-4ef3-a467-243f61da2fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:00.204374 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9c270b-59c2-4407-a350-53179cf8e724 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:00.222069 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:00.233658 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-06b46f5f-5a89-4ef3-a467-243f61da2fcf None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:32:00.234858 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-06b46f5f-5a89-4ef3-a467-243f61da2fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:32:00.301265 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:00.346584 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 389/779] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:32:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:32:00.372314 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b46f5f-5a89-4ef3-a467-243f61da2fcf None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:00.390448 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9c270b-59c2-4407-a350-53179cf8e724 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:00.425844 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9edd36f-0f70-446c-a81c-82e6fe366997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:00.426851 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9edd36f-0f70-446c-a81c-82e6fe366997 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:00.427337 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9edd36f-0f70-446c-a81c-82e6fe366997 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:00.427920 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9edd36f-0f70-446c-a81c-82e6fe366997 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:00.429132 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9edd36f-0f70-446c-a81c-82e6fe366997 None admin] RBAC: Adding query filter params (type=network) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:00.430431 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9edd36f-0f70-446c-a81c-82e6fe366997 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:00.432855 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9edd36f-0f70-446c-a81c-82e6fe366997 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:00.445193 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 391/780] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Jun 15 14:32:00 2022] GET /identity/v3/services?type=network => generated 350 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:32:00.453042 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9edd36f-0f70-446c-a81c-82e6fe366997 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:00.536762 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b46f5f-5a89-4ef3-a467-243f61da2fcf None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:00.600441 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:00.612544 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93cd50f-e520-43b8-afac-f18772dff5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:00.613347 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93cd50f-e520-43b8-afac-f18772dff5df None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:00.613634 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93cd50f-e520-43b8-afac-f18772dff5df None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:00.613917 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93cd50f-e520-43b8-afac-f18772dff5df None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:00.635477 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93cd50f-e520-43b8-afac-f18772dff5df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:32:00.646603 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9edd36f-0f70-446c-a81c-82e6fe366997 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:00.650587 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-b93cd50f-e520-43b8-afac-f18772dff5df None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:32:00.650587 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:00.650587 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:00.650587 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:00.650587 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:00.650587 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:00.650587 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:00.650587 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:00.650587 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:00.650587 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:00.650587 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:00.650587 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:00.650587 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:32:00.650587 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:32:00.650587 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:32:00.650587 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:32:00.651535 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:32:00.651535 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:32:00.651535 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:32:00.651535 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:32:00.651535 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:00.651535 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:00.651535 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:00.651535 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:00.651535 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:00.651535 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:00.651535 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:00.651535 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:00.651535 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:00.651535 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:00.651535 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:00.651535 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:00.652482 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:00.652482 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:00.652482 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:00.652482 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:00.652482 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:32:00.652482 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:32:00.652482 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:32:00.652482 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:32:00.652482 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:32:00.652482 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:32:00.652482 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:32:00.652482 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:32:00.654046 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 390/781] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jun 15 14:32:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:00.661021 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93cd50f-e520-43b8-afac-f18772dff5df None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:00.715863 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f95a212-68e8-4e55-a88e-65ec0d9acfa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:00.716457 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:00.716671 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f95a212-68e8-4e55-a88e-65ec0d9acfa4 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:00.716831 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f95a212-68e8-4e55-a88e-65ec0d9acfa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:00.717104 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f95a212-68e8-4e55-a88e-65ec0d9acfa4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:00.717926 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f95a212-68e8-4e55-a88e-65ec0d9acfa4 None admin] RBAC: Adding query filter params (interface=public, service_id=a2fee84d45fb44d78abe636cca567512, region_id=RegionOne) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:00.718246 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f95a212-68e8-4e55-a88e-65ec0d9acfa4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:00.719673 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f95a212-68e8-4e55-a88e-65ec0d9acfa4 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:00.734716 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 392/782] 10.222.0.42 () {56 vars in 1251 bytes} [Wed Jun 15 14:32:00 2022] GET /identity/v3/endpoints?service_id=a2fee84d45fb44d78abe636cca567512&interface=public®ion_id=RegionOne => generated 194 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:00.741461 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f95a212-68e8-4e55-a88e-65ec0d9acfa4 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:00.803324 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93cd50f-e520-43b8-afac-f18772dff5df None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:00.838613 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e02697-d739-4902-a31f-a13c5c871b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:00.839295 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e02697-d739-4902-a31f-a13c5c871b0a None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:00.839894 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e02697-d739-4902-a31f-a13c5c871b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:00.840238 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e02697-d739-4902-a31f-a13c5c871b0a None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:00.840782 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e02697-d739-4902-a31f-a13c5c871b0a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:00.841206 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e02697-d739-4902-a31f-a13c5c871b0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:00.842641 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e02697-d739-4902-a31f-a13c5c871b0a None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:00.856412 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 391/783] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jun 15 14:32:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:00.858549 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e02697-d739-4902-a31f-a13c5c871b0a None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:00.873194 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f95a212-68e8-4e55-a88e-65ec0d9acfa4 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:00.936393 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75785a4c-4f97-46ea-9097-5b236e28ea7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:00.936984 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75785a4c-4f97-46ea-9097-5b236e28ea7d None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:00.937315 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75785a4c-4f97-46ea-9097-5b236e28ea7d None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:00.937638 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75785a4c-4f97-46ea-9097-5b236e28ea7d None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:00.938321 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75785a4c-4f97-46ea-9097-5b236e28ea7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:00.939915 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75785a4c-4f97-46ea-9097-5b236e28ea7d None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:00.944770 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:00.950041 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 393/784] 10.222.0.42 () {56 vars in 1087 bytes} [Wed Jun 15 14:32:00 2022] GET /identity/v3/services => generated 772 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:32:00.966058 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:00.976026 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e02697-d739-4902-a31f-a13c5c871b0a None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:01.015494 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbd7d40-ab78-4a5b-b1c7-65ed4ad43a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:01.016340 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd7d40-ab78-4a5b-b1c7-65ed4ad43a79 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:01.016624 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd7d40-ab78-4a5b-b1c7-65ed4ad43a79 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:01.016904 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd7d40-ab78-4a5b-b1c7-65ed4ad43a79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:01.029324 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbd7d40-ab78-4a5b-b1c7-65ed4ad43a79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:01.030811 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbd7d40-ab78-4a5b-b1c7-65ed4ad43a79 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:01.048771 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-abbd7d40-ab78-4a5b-b1c7-65ed4ad43a79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:01.048771 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:01.048771 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:01.048771 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:01.048771 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:01.048771 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:01.048771 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:01.048771 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:01.048771 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:01.048771 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:01.048771 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:01.048771 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:01.048771 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:32:01.048771 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:32:01.048771 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:32:01.050922 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:32:01.050922 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:01.050922 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:01.050922 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:01.050922 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:01.050922 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:01.050922 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:01.050922 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:01.050922 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:01.050922 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:01.050922 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:01.050922 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:01.050922 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:01.050922 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:01.050922 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:01.050922 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:01.050922 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:01.053134 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:32:01.053134 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:32:01.053134 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:01.053134 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:32:01.053134 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 392/785] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:32:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:01.056010 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75785a4c-4f97-46ea-9097-5b236e28ea7d None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:01.190455 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75785a4c-4f97-46ea-9097-5b236e28ea7d None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:01.227231 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca13896-7d52-4c15-a5eb-aa5726d6e3f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:01.227906 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca13896-7d52-4c15-a5eb-aa5726d6e3f1 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:01.228221 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca13896-7d52-4c15-a5eb-aa5726d6e3f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:01.228503 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca13896-7d52-4c15-a5eb-aa5726d6e3f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:01.229227 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca13896-7d52-4c15-a5eb-aa5726d6e3f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:01.229615 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca13896-7d52-4c15-a5eb-aa5726d6e3f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:01.230997 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca13896-7d52-4c15-a5eb-aa5726d6e3f1 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:01.247377 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 394/786] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:32:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:01.252947 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbd7d40-ab78-4a5b-b1c7-65ed4ad43a79 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:01.332237 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d6a214-224f-4132-ac32-009a47132ce5 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:01.332577 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d6a214-224f-4132-ac32-009a47132ce5 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:01.332790 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d6a214-224f-4132-ac32-009a47132ce5 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:01.334477 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 395/787] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:32:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:32:01.339425 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0485109a-2168-463f-bdd2-ab2a701f289d None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:01.339854 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0485109a-2168-463f-bdd2-ab2a701f289d None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:01.342352 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0485109a-2168-463f-bdd2-ab2a701f289d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:01.359186 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-0485109a-2168-463f-bdd2-ab2a701f289d None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:32:01.360397 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-0485109a-2168-463f-bdd2-ab2a701f289d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:32:01.422627 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:01.435689 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 396/788] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:32:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:32:01.460323 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0485109a-2168-463f-bdd2-ab2a701f289d None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:01.493788 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbd7d40-ab78-4a5b-b1c7-65ed4ad43a79 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:01.567554 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2af919-8577-4cc9-aeea-a34fd3879801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:01.568996 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2af919-8577-4cc9-aeea-a34fd3879801 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:01.569433 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2af919-8577-4cc9-aeea-a34fd3879801 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:01.569887 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2af919-8577-4cc9-aeea-a34fd3879801 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:01.583154 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2af919-8577-4cc9-aeea-a34fd3879801 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:01.584876 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2af919-8577-4cc9-aeea-a34fd3879801 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:01.589479 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0485109a-2168-463f-bdd2-ab2a701f289d None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:01.609192 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-5f2af919-8577-4cc9-aeea-a34fd3879801 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 15 14:32:01.609192 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:01.609192 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:01.609192 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:01.609192 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:01.609192 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:01.609192 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:01.609192 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:01.609192 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:01.609192 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:01.609192 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:01.609192 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:01.609192 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:32:01.609192 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:32:01.609192 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:32:01.609192 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:32:01.610420 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:01.610420 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:01.610420 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:32:01.610420 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:32:01.610420 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:32:01.610420 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:32:01.610420 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:01.610420 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:01.610420 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:01.610420 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:01.610420 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:01.610420 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:01.610420 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:01.610420 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:01.610420 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:01.610420 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:01.610420 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:01.610420 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:01.611612 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:01.611612 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:01.611612 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:32:01.611612 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:32:01.611612 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:32:01.611612 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:32:01.611612 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:32:01.611612 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:32:01.611612 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 15 14:32:01.611612 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:32:01.613900 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 393/789] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jun 15 14:32:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:01.619468 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2af919-8577-4cc9-aeea-a34fd3879801 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:01.623182 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:01.632027 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6227a2-3076-4bd8-8c3c-a922060c5825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:01.636647 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6227a2-3076-4bd8-8c3c-a922060c5825 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:01.636647 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6227a2-3076-4bd8-8c3c-a922060c5825 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:01.636647 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6227a2-3076-4bd8-8c3c-a922060c5825 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:01.645617 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6227a2-3076-4bd8-8c3c-a922060c5825 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:32:01.654816 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-ec6227a2-3076-4bd8-8c3c-a922060c5825 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:32:01.654816 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:01.654816 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:01.654816 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:01.654816 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:01.654816 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:01.654816 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:01.654816 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:01.654816 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:01.654816 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:01.654816 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:01.654816 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:01.654816 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:32:01.654816 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:32:01.654816 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:32:01.654816 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:32:01.655831 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:32:01.655831 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:32:01.655831 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:32:01.655831 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:32:01.655831 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:01.655831 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:01.655831 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:01.655831 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:01.655831 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:01.655831 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:01.655831 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:01.655831 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:01.655831 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:01.655831 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:01.655831 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:01.655831 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:01.656784 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:01.656784 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:01.656784 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:01.656784 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:01.656784 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:32:01.656784 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:32:01.656784 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:32:01.656784 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:32:01.656784 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:32:01.656784 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:32:01.656784 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:32:01.656784 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:01.659487 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 397/790] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jun 15 14:32:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:01.666429 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6227a2-3076-4bd8-8c3c-a922060c5825 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:01.721976 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:01.777670 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6227a2-3076-4bd8-8c3c-a922060c5825 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:01.815278 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31110b05-baaf-4e41-a8aa-fa8c16d7d853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:01.816353 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31110b05-baaf-4e41-a8aa-fa8c16d7d853 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:01.816881 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31110b05-baaf-4e41-a8aa-fa8c16d7d853 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:01.817352 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31110b05-baaf-4e41-a8aa-fa8c16d7d853 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:01.818285 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31110b05-baaf-4e41-a8aa-fa8c16d7d853 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:01.818810 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31110b05-baaf-4e41-a8aa-fa8c16d7d853 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:01.820394 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31110b05-baaf-4e41-a8aa-fa8c16d7d853 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:01.835636 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 398/791] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jun 15 14:32:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:01.842352 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31110b05-baaf-4e41-a8aa-fa8c16d7d853 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:01.910045 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2af919-8577-4cc9-aeea-a34fd3879801 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:01.970040 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31110b05-baaf-4e41-a8aa-fa8c16d7d853 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:01.986918 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:02.014172 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70169a1b-37aa-4447-9c95-198f29fbd56f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:02.015371 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70169a1b-37aa-4447-9c95-198f29fbd56f None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:02.015855 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70169a1b-37aa-4447-9c95-198f29fbd56f None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:02.016429 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70169a1b-37aa-4447-9c95-198f29fbd56f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:02.020885 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc83bc3-6214-4446-9a05-56ea25e9ca4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:02.021588 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc83bc3-6214-4446-9a05-56ea25e9ca4b None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:02.021869 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc83bc3-6214-4446-9a05-56ea25e9ca4b None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:02.022148 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc83bc3-6214-4446-9a05-56ea25e9ca4b None admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:02.023007 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc83bc3-6214-4446-9a05-56ea25e9ca4b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:02.023319 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc83bc3-6214-4446-9a05-56ea25e9ca4b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:02.025074 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc83bc3-6214-4446-9a05-56ea25e9ca4b None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:02.032964 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70169a1b-37aa-4447-9c95-198f29fbd56f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:02.035961 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70169a1b-37aa-4447-9c95-198f29fbd56f None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:02.059157 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-70169a1b-37aa-4447-9c95-198f29fbd56f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:02.059157 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:02.059157 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:02.059157 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:02.059157 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:02.059157 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:02.059157 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:02.059157 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:02.059157 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:02.059157 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:02.059157 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:02.059157 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:02.059157 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:32:02.059157 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:32:02.059157 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:32:02.060150 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:32:02.060150 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:02.060150 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:02.060150 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:02.060150 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:02.060150 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:02.060150 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:02.060150 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:02.060150 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:02.060150 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:02.060150 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:02.060150 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:02.060150 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:02.060150 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:02.060150 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:02.060150 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:02.060150 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:02.061058 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:32:02.061058 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:32:02.061058 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:02.061058 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:02.064043 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 399/792] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:32:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:32:02.069927 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70169a1b-37aa-4447-9c95-198f29fbd56f None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:02.078409 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 394/793] 10.222.0.42 () {56 vars in 1135 bytes} [Wed Jun 15 14:32:01 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 459 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:32:02.085098 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc83bc3-6214-4446-9a05-56ea25e9ca4b None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:02.168694 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:02.196411 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70169a1b-37aa-4447-9c95-198f29fbd56f None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:02.216842 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc83bc3-6214-4446-9a05-56ea25e9ca4b None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:02.230141 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2205932-8c7c-4069-8c67-d1d70114f4d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:02.231324 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2205932-8c7c-4069-8c67-d1d70114f4d9 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:02.231599 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2205932-8c7c-4069-8c67-d1d70114f4d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:02.231877 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2205932-8c7c-4069-8c67-d1d70114f4d9 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:02.232622 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2205932-8c7c-4069-8c67-d1d70114f4d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:02.232930 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2205932-8c7c-4069-8c67-d1d70114f4d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:02.234386 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2205932-8c7c-4069-8c67-d1d70114f4d9 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:02.251910 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 400/794] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:32:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:32:02.251910 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bf8c22-d460-44aa-98fa-366e9341271c None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:02.251910 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bf8c22-d460-44aa-98fa-366e9341271c None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:02.251910 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bf8c22-d460-44aa-98fa-366e9341271c None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:02.251910 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 401/795] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:32:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:32:02.252713 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bf8c22-d460-44aa-98fa-366e9341271c None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:02.263439 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15dcf84-de04-4071-87e1-a04394ff2138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:02.264708 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15dcf84-de04-4071-87e1-a04394ff2138 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:02.265193 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15dcf84-de04-4071-87e1-a04394ff2138 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:02.265851 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15dcf84-de04-4071-87e1-a04394ff2138 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:02.281430 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:02.281779 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15dcf84-de04-4071-87e1-a04394ff2138 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:02.284386 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15dcf84-de04-4071-87e1-a04394ff2138 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:02.305178 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-a15dcf84-de04-4071-87e1-a04394ff2138 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:02.305178 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:02.305178 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:02.305178 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:02.305178 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:02.305178 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:02.305178 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:02.305178 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:02.305178 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:02.305178 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:02.305178 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:02.305178 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:02.305178 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:32:02.305178 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:32:02.305178 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:32:02.306158 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:32:02.306158 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:02.306158 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:02.306158 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:02.306158 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:02.306158 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:02.306158 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:02.306158 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:02.306158 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:02.306158 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:02.306158 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:02.306158 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:02.306158 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:02.306158 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:02.306158 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:02.306158 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:02.306158 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:02.307077 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:32:02.307077 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:32:02.307077 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:02.307077 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:32:02.309968 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 395/796] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:32:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:02.311654 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c348e3-4f2f-4273-b982-922a9fb4b17d None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:02.312172 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c348e3-4f2f-4273-b982-922a9fb4b17d None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:02.312651 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c348e3-4f2f-4273-b982-922a9fb4b17d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:02.331877 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-18c348e3-4f2f-4273-b982-922a9fb4b17d None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:32:02.333571 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-18c348e3-4f2f-4273-b982-922a9fb4b17d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:32:02.364180 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bf8c22-d460-44aa-98fa-366e9341271c None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:02.400468 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f26954-e5ee-4003-a82e-6d8c205095f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:02.401256 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f26954-e5ee-4003-a82e-6d8c205095f1 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:02.401531 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f26954-e5ee-4003-a82e-6d8c205095f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:02.401803 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f26954-e5ee-4003-a82e-6d8c205095f1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:02.412400 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 396/797] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:32:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:32:02.412400 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c348e3-4f2f-4273-b982-922a9fb4b17d None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:02.413951 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f26954-e5ee-4003-a82e-6d8c205095f1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:02.415320 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f26954-e5ee-4003-a82e-6d8c205095f1 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:02.438967 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-f5f26954-e5ee-4003-a82e-6d8c205095f1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:32:02.438967 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:02.438967 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:02.438967 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:02.438967 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:02.438967 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:02.438967 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:02.438967 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:02.438967 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:02.438967 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:02.438967 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:02.438967 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:02.438967 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:32:02.438967 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:32:02.438967 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:32:02.438967 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:32:02.439983 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:02.439983 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:02.439983 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:32:02.439983 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:32:02.439983 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:32:02.439983 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:32:02.439983 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:02.439983 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:02.439983 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:02.439983 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:02.439983 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:02.439983 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:02.439983 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:02.439983 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:02.439983 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:02.439983 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:02.439983 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:02.439983 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:02.440977 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:02.440977 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:02.440977 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:32:02.440977 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:32:02.440977 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:32:02.440977 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:32:02.440977 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:32:02.440977 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:32:02.440977 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:32:02.440977 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:02.440977 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 402/798] 10.222.0.42 () {56 vars in 1093 bytes} [Wed Jun 15 14:32:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:02.442107 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa021fba-f492-4358-bcad-0de6b0deb7b9 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:02.442107 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa021fba-f492-4358-bcad-0de6b0deb7b9 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:02.442107 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa021fba-f492-4358-bcad-0de6b0deb7b9 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:02.444509 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 403/799] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:32:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:32:02.444509 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa021fba-f492-4358-bcad-0de6b0deb7b9 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:02.465982 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:02.529866 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c348e3-4f2f-4273-b982-922a9fb4b17d None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:02.545861 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:02.562477 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa021fba-f492-4358-bcad-0de6b0deb7b9 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:02.574700 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcfc130-1b3d-4836-b1b4-7e5d6e44337b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:02.575532 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcfc130-1b3d-4836-b1b4-7e5d6e44337b None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:02.575899 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcfc130-1b3d-4836-b1b4-7e5d6e44337b None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:02.576270 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcfc130-1b3d-4836-b1b4-7e5d6e44337b None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:02.577060 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcfc130-1b3d-4836-b1b4-7e5d6e44337b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:02.577444 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcfc130-1b3d-4836-b1b4-7e5d6e44337b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:02.578898 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcfc130-1b3d-4836-b1b4-7e5d6e44337b None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:02.595777 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 397/800] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:32:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:02.595777 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcfc130-1b3d-4836-b1b4-7e5d6e44337b None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:02.605008 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc9c86a-4399-48f8-9641-42b64e8f3adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:02.606522 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc9c86a-4399-48f8-9641-42b64e8f3adb None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:02.606802 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc9c86a-4399-48f8-9641-42b64e8f3adb None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:02.607074 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc9c86a-4399-48f8-9641-42b64e8f3adb None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:02.607782 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc9c86a-4399-48f8-9641-42b64e8f3adb None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:02.609114 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc9c86a-4399-48f8-9641-42b64e8f3adb None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:02.621785 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-4dc9c86a-4399-48f8-9641-42b64e8f3adb None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 14:32:02.621785 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:02.621785 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:32:02.621785 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:32:02.621785 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:32:02.621785 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:32:02.621785 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:32:02.621785 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:32:02.621785 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 14:32:02.621785 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:02.621785 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:32:02.621785 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:02.621785 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:02.621785 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:02.621785 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:02.621785 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:02.622926 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:02.622926 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:02.622926 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:02.622926 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:02.622926 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:02.622926 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:02.622926 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:02.622926 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:32:02.622926 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:32:02.622926 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:32:02.622926 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:32:02.622926 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:02.622926 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:02.622926 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:02.622926 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:02.622926 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:02.623757 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:02.623757 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:02.623757 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:02.623757 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:02.623757 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:02.623757 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:02.623757 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:02.623757 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:02.623757 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:02.623757 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:02.623757 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:02.623757 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:32:02.623757 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:32:02.623757 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 14:32:02.623757 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:02.623757 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 404/801] 10.222.0.42 () {56 vars in 1129 bytes} [Wed Jun 15 14:32:02 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:02.624661 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348a2dd2-08f5-4246-b7b6-7110395002ab None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:02.624661 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348a2dd2-08f5-4246-b7b6-7110395002ab None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:02.624793 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348a2dd2-08f5-4246-b7b6-7110395002ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:02.641672 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-348a2dd2-08f5-4246-b7b6-7110395002ab None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:32:02.642845 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-348a2dd2-08f5-4246-b7b6-7110395002ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:32:02.705980 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcfc130-1b3d-4836-b1b4-7e5d6e44337b None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:02.707887 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:02.712867 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 405/802] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:32:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:32:02.714238 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348a2dd2-08f5-4246-b7b6-7110395002ab None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:02.743219 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b54ed-3ea9-49bb-9469-d2549024411a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:02.743887 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b54ed-3ea9-49bb-9469-d2549024411a None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:02.744196 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b54ed-3ea9-49bb-9469-d2549024411a None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:02.744470 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b54ed-3ea9-49bb-9469-d2549024411a None admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:02.745297 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0b54ed-3ea9-49bb-9469-d2549024411a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:02.745600 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0b54ed-3ea9-49bb-9469-d2549024411a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:02.747208 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0b54ed-3ea9-49bb-9469-d2549024411a None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:02.770246 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 398/803] 10.222.0.42 () {56 vars in 1133 bytes} [Wed Jun 15 14:32:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:02.770246 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b54ed-3ea9-49bb-9469-d2549024411a None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:02.803179 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:02.825964 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348a2dd2-08f5-4246-b7b6-7110395002ab None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:02.861647 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9779fbff-60ef-49f8-a593-4cc35975742c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:02.862451 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9779fbff-60ef-49f8-a593-4cc35975742c None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:02.862735 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9779fbff-60ef-49f8-a593-4cc35975742c None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:02.863014 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9779fbff-60ef-49f8-a593-4cc35975742c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:02.869937 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9779fbff-60ef-49f8-a593-4cc35975742c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:02.872116 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9779fbff-60ef-49f8-a593-4cc35975742c None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:02.878052 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b54ed-3ea9-49bb-9469-d2549024411a None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:02.885773 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-9779fbff-60ef-49f8-a593-4cc35975742c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:32:02.885773 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:02.885773 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:02.885773 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:02.885773 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:02.885773 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:02.885773 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:02.885773 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:02.885773 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:02.885773 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:02.885773 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:02.885773 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:02.885773 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:32:02.885773 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:32:02.885773 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:32:02.886956 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:32:02.886956 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:02.886956 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:02.886956 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:02.886956 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:02.886956 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:02.886956 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:02.886956 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:02.886956 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:02.886956 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:02.886956 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:02.886956 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:02.886956 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:02.886956 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:02.886956 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:02.886956 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:02.886956 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:02.893165 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:32:02.893165 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:32:02.893165 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:32:02.893165 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:32:02.893165 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:32:02.893165 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:32:02.893165 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:32:02.893165 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:02.893165 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 406/804] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jun 15 14:32:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:02.893165 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9779fbff-60ef-49f8-a593-4cc35975742c None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:02.921267 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6261ac14-375a-4f04-bcdb-d9a5a3ccfaa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:02.922086 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6261ac14-375a-4f04-bcdb-d9a5a3ccfaa9 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:02.922664 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6261ac14-375a-4f04-bcdb-d9a5a3ccfaa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:02.922949 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6261ac14-375a-4f04-bcdb-d9a5a3ccfaa9 None admin] PATH_INFO: `/v3/services` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:02.923793 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6261ac14-375a-4f04-bcdb-d9a5a3ccfaa9 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:02.924120 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6261ac14-375a-4f04-bcdb-d9a5a3ccfaa9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:02.925473 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6261ac14-375a-4f04-bcdb-d9a5a3ccfaa9 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:02.930554 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 399/805] 10.222.0.42 () {56 vars in 1124 bytes} [Wed Jun 15 14:32:02 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:02.931168 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6261ac14-375a-4f04-bcdb-d9a5a3ccfaa9 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:02.958052 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:03.001339 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9779fbff-60ef-49f8-a593-4cc35975742c None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:03.035367 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea28ff5-d288-45ca-b063-0dd96aeedca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:03.036187 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea28ff5-d288-45ca-b063-0dd96aeedca9 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:03.036472 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea28ff5-d288-45ca-b063-0dd96aeedca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:03.036749 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea28ff5-d288-45ca-b063-0dd96aeedca9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:03.038782 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6261ac14-375a-4f04-bcdb-d9a5a3ccfaa9 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:03.044661 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:03.049662 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea28ff5-d288-45ca-b063-0dd96aeedca9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:32:03.057714 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-fea28ff5-d288-45ca-b063-0dd96aeedca9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:32:03.057714 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:03.057714 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:03.057714 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:03.057714 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:03.057714 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:03.057714 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:03.057714 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:03.057714 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:03.057714 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:03.057714 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:03.057714 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:03.057714 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:32:03.057714 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:32:03.057714 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:32:03.057714 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:32:03.059194 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:32:03.059194 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:32:03.059194 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:32:03.059194 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:32:03.059194 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:03.059194 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:03.059194 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:03.059194 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:03.059194 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:03.059194 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:03.059194 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:03.059194 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:03.059194 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:03.059194 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:03.059194 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:03.059194 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:03.065490 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:03.065490 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:03.065490 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:03.065490 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:03.065490 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:32:03.065490 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:32:03.065490 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:32:03.065490 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:32:03.065490 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:32:03.065490 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:32:03.065490 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:32:03.065490 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:03.065490 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 407/806] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Jun 15 14:32:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:03.065490 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea28ff5-d288-45ca-b063-0dd96aeedca9 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:03.085281 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e816830e-7104-4b26-ba58-d1d6d5c728e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:03.086163 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e816830e-7104-4b26-ba58-d1d6d5c728e1 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:03.086449 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e816830e-7104-4b26-ba58-d1d6d5c728e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:03.086772 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e816830e-7104-4b26-ba58-d1d6d5c728e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:03.159375 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e816830e-7104-4b26-ba58-d1d6d5c728e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:03.160876 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e816830e-7104-4b26-ba58-d1d6d5c728e1 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:03.170735 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea28ff5-d288-45ca-b063-0dd96aeedca9 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:03.186503 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:03.206044 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38ccbdb-78bb-4ade-94de-0600dab8966b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:03.206710 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38ccbdb-78bb-4ade-94de-0600dab8966b None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:03.211571 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38ccbdb-78bb-4ade-94de-0600dab8966b None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:03.211571 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38ccbdb-78bb-4ade-94de-0600dab8966b None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:03.211571 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38ccbdb-78bb-4ade-94de-0600dab8966b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:03.211571 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38ccbdb-78bb-4ade-94de-0600dab8966b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:03.211571 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38ccbdb-78bb-4ade-94de-0600dab8966b None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:03.222875 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 408/807] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jun 15 14:32:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:03.223764 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38ccbdb-78bb-4ade-94de-0600dab8966b None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:03.270908 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 400/808] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jun 15 14:32:02 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:32:03.272238 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e816830e-7104-4b26-ba58-d1d6d5c728e1 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:03.291138 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:03.336229 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38ccbdb-78bb-4ade-94de-0600dab8966b None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:03.378727 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554d29ad-e89c-42c5-a631-43a1d2e65a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:03.379672 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554d29ad-e89c-42c5-a631-43a1d2e65a4a None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:03.379962 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554d29ad-e89c-42c5-a631-43a1d2e65a4a None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:03.380269 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554d29ad-e89c-42c5-a631-43a1d2e65a4a None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:03.380973 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554d29ad-e89c-42c5-a631-43a1d2e65a4a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:03.381283 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554d29ad-e89c-42c5-a631-43a1d2e65a4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:03.381752 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e816830e-7104-4b26-ba58-d1d6d5c728e1 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:03.382635 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554d29ad-e89c-42c5-a631-43a1d2e65a4a None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:03.393789 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 409/809] 10.222.0.42 () {56 vars in 1124 bytes} [Wed Jun 15 14:32:03 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:03.393789 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554d29ad-e89c-42c5-a631-43a1d2e65a4a None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:03.410313 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:03.422997 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1639589-d47e-4be9-be90-d904cf4e7675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:03.423689 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1639589-d47e-4be9-be90-d904cf4e7675 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:03.425418 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1639589-d47e-4be9-be90-d904cf4e7675 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:03.425418 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1639589-d47e-4be9-be90-d904cf4e7675 None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:03.425418 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1639589-d47e-4be9-be90-d904cf4e7675 None admin] RBAC: Adding query filter params (name=service) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:03.425418 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1639589-d47e-4be9-be90-d904cf4e7675 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:03.426869 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1639589-d47e-4be9-be90-d904cf4e7675 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:03.440864 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 401/810] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Jun 15 14:32:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:03.440864 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1639589-d47e-4be9-be90-d904cf4e7675 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:03.513392 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554d29ad-e89c-42c5-a631-43a1d2e65a4a None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:03.535952 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:03.548402 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b046a563-d17f-4f04-9a0b-315ba8e6722f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:03.551414 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1639589-d47e-4be9-be90-d904cf4e7675 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:03.551529 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b046a563-d17f-4f04-9a0b-315ba8e6722f None admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:03.551529 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b046a563-d17f-4f04-9a0b-315ba8e6722f None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:03.551529 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b046a563-d17f-4f04-9a0b-315ba8e6722f None admin] PATH_INFO: `/v3/projects/3094ff8966f149bda784da6a51b1358f/users/c04dc5832333498e96a7e02bbfb9484c/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:03.552177 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b046a563-d17f-4f04-9a0b-315ba8e6722f None admin] RBAC: Authorizing `identity:create_grant(project_id=3094ff8966f149bda784da6a51b1358f, user_id=c04dc5832333498e96a7e02bbfb9484c, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:03.553642 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b046a563-d17f-4f04-9a0b-315ba8e6722f None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:03.582367 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e542c8-2d61-4cea-899c-530458bbc24f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:03.583149 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e542c8-2d61-4cea-899c-530458bbc24f None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:03.583429 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e542c8-2d61-4cea-899c-530458bbc24f None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:03.583704 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e542c8-2d61-4cea-899c-530458bbc24f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:03.593999 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e542c8-2d61-4cea-899c-530458bbc24f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:03.595376 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e542c8-2d61-4cea-899c-530458bbc24f None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:03.612361 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-69e542c8-2d61-4cea-899c-530458bbc24f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:03.612361 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:03.612361 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:03.612361 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:03.612361 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:03.612361 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:03.612361 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:03.612361 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:03.612361 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:03.612361 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:03.612361 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:03.612361 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:03.612361 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:32:03.612361 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:32:03.612361 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:32:03.613408 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:32:03.613408 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:03.613408 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:03.613408 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:03.613408 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:03.613408 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:03.613408 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:03.613408 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:03.613408 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:03.613408 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:03.613408 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:03.613408 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:03.613408 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:03.613408 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:03.613408 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:03.613408 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:03.613408 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:03.614437 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:32:03.614437 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:32:03.614437 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:03.614437 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:32:03.614437 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 402/811] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:32:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:03.614437 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e542c8-2d61-4cea-899c-530458bbc24f None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:03.626990 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 410/812] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Jun 15 14:32:03 2022] PUT /identity/v3/projects/3094ff8966f149bda784da6a51b1358f/users/c04dc5832333498e96a7e02bbfb9484c/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:32:03.627886 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b046a563-d17f-4f04-9a0b-315ba8e6722f None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:03.667210 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:03.736392 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e542c8-2d61-4cea-899c-530458bbc24f None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:03.749307 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b046a563-d17f-4f04-9a0b-315ba8e6722f None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:03.776661 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ced679-1762-4f43-8672-881e3769b255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:03.777543 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ced679-1762-4f43-8672-881e3769b255 None admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:03.777930 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ced679-1762-4f43-8672-881e3769b255 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:03.778212 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ced679-1762-4f43-8672-881e3769b255 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:03.778971 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ced679-1762-4f43-8672-881e3769b255 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:03.780378 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ced679-1762-4f43-8672-881e3769b255 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:03.794992 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4fed43-2246-4f61-90ef-bd6dc0949f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:03.795821 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4fed43-2246-4f61-90ef-bd6dc0949f1e None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:03.796128 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4fed43-2246-4f61-90ef-bd6dc0949f1e None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:03.796413 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4fed43-2246-4f61-90ef-bd6dc0949f1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:03.809618 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4fed43-2246-4f61-90ef-bd6dc0949f1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:03.811112 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4fed43-2246-4f61-90ef-bd6dc0949f1e None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:03.828926 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-6c4fed43-2246-4f61-90ef-bd6dc0949f1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:03.828926 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:03.828926 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:03.828926 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:03.828926 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:03.828926 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:03.828926 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:03.828926 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:03.828926 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:03.828926 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:03.828926 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:03.828926 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:03.828926 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:32:03.828926 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:32:03.828926 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:32:03.829994 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:32:03.829994 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:03.829994 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:03.829994 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:03.829994 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:03.829994 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:03.829994 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:03.829994 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:03.829994 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:03.829994 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:03.829994 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:03.829994 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:03.829994 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:03.829994 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:03.829994 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:03.829994 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:03.829994 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:03.830968 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:32:03.830968 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:32:03.830968 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:03.830968 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:03.830968 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 411/813] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:32:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:03.831560 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d82dd9d-3abd-4d4e-8358-54da40c724b8 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:03.831774 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d82dd9d-3abd-4d4e-8358-54da40c724b8 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:03.832054 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d82dd9d-3abd-4d4e-8358-54da40c724b8 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:03.837707 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 412/814] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:32:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:32:03.837707 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d82dd9d-3abd-4d4e-8358-54da40c724b8 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:03.873371 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:03.899905 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 403/815] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Jun 15 14:32:03 2022] POST /identity/v3/endpoints => generated 346 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:32:03.900899 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ced679-1762-4f43-8672-881e3769b255 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:03.959591 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d82dd9d-3abd-4d4e-8358-54da40c724b8 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:04.005019 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8781a94f-df2b-4ff1-9038-e59a133b804c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:04.005774 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8781a94f-df2b-4ff1-9038-e59a133b804c None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:04.006147 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8781a94f-df2b-4ff1-9038-e59a133b804c None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:04.006511 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8781a94f-df2b-4ff1-9038-e59a133b804c None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:04.007585 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8781a94f-df2b-4ff1-9038-e59a133b804c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:04.008977 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8781a94f-df2b-4ff1-9038-e59a133b804c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:04.010470 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8781a94f-df2b-4ff1-9038-e59a133b804c None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:04.028504 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 413/816] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:32:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:04.030807 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6426a61-edb7-47e8-8a96-f62cf874f080 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:04.031217 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6426a61-edb7-47e8-8a96-f62cf874f080 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:04.031575 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6426a61-edb7-47e8-8a96-f62cf874f080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:04.044290 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:04.053408 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ced679-1762-4f43-8672-881e3769b255 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:04.061170 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-c6426a61-edb7-47e8-8a96-f62cf874f080 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:32:04.062739 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c6426a61-edb7-47e8-8a96-f62cf874f080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:32:04.097454 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aba4dfc-0020-4ba4-88aa-c55f423e4232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:04.098125 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aba4dfc-0020-4ba4-88aa-c55f423e4232 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:04.098413 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aba4dfc-0020-4ba4-88aa-c55f423e4232 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:04.098692 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aba4dfc-0020-4ba4-88aa-c55f423e4232 None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:04.099410 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aba4dfc-0020-4ba4-88aa-c55f423e4232 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:04.099945 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aba4dfc-0020-4ba4-88aa-c55f423e4232 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:04.101415 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aba4dfc-0020-4ba4-88aa-c55f423e4232 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:04.116935 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 404/817] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:32:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:04.117889 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aba4dfc-0020-4ba4-88aa-c55f423e4232 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:04.152691 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 414/818] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:32:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:32:04.153759 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6426a61-edb7-47e8-8a96-f62cf874f080 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:04.184637 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:04.232356 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aba4dfc-0020-4ba4-88aa-c55f423e4232 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:04.277803 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c6416f-9f86-468b-af2b-56b74c2925c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:04.278817 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c6416f-9f86-468b-af2b-56b74c2925c1 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:04.279103 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c6416f-9f86-468b-af2b-56b74c2925c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:04.279385 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c6416f-9f86-468b-af2b-56b74c2925c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:04.280188 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6426a61-edb7-47e8-8a96-f62cf874f080 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:04.288445 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c6416f-9f86-468b-af2b-56b74c2925c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:04.290705 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c6416f-9f86-468b-af2b-56b74c2925c1 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:04.307618 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-c2c6416f-9f86-468b-af2b-56b74c2925c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:32:04.307618 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:04.307618 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:04.307618 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:04.307618 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:04.307618 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:04.307618 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:04.307618 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:04.307618 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:04.307618 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:04.307618 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:04.307618 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:04.307618 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:32:04.307618 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:32:04.307618 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:32:04.309192 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:32:04.309192 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:04.309192 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:04.309192 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:04.309192 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:04.309192 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:04.309192 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:04.309192 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:04.309192 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:04.309192 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:04.309192 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:04.309192 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:04.309192 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:04.309192 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:04.309192 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:04.309192 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:04.309192 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:04.310090 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:32:04.310090 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:32:04.310090 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:32:04.310090 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:32:04.310090 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:32:04.310090 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:32:04.310090 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:32:04.310090 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:32:04.314581 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 405/819] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jun 15 14:32:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:04.315575 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c6416f-9f86-468b-af2b-56b74c2925c1 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:04.325587 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfdaca6-fd2b-4fe0-8a57-a4a8d5157bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:04.327191 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfdaca6-fd2b-4fe0-8a57-a4a8d5157bc3 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:04.327674 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfdaca6-fd2b-4fe0-8a57-a4a8d5157bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:04.328153 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfdaca6-fd2b-4fe0-8a57-a4a8d5157bc3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:04.336499 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:04.338691 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfdaca6-fd2b-4fe0-8a57-a4a8d5157bc3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:04.340930 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfdaca6-fd2b-4fe0-8a57-a4a8d5157bc3 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:04.360896 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-4bfdaca6-fd2b-4fe0-8a57-a4a8d5157bc3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 14:32:04.360896 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:04.360896 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:04.360896 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:04.360896 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:04.360896 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:04.360896 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:04.360896 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:04.360896 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:04.360896 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:04.360896 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:04.360896 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:04.360896 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:32:04.360896 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:32:04.360896 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:32:04.360896 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:32:04.361869 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:04.361869 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:04.361869 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:32:04.361869 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:32:04.361869 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:32:04.361869 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:32:04.361869 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:04.361869 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:04.361869 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:04.361869 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:04.361869 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:04.361869 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:04.361869 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:04.361869 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:04.361869 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:04.361869 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:04.361869 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:04.361869 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:04.362902 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:04.362902 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:04.362902 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:32:04.362902 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:32:04.362902 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:32:04.362902 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:32:04.362902 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:32:04.362902 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:32:04.362902 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 14:32:04.362902 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:04.366066 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 415/820] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Jun 15 14:32:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:04.367184 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfdaca6-fd2b-4fe0-8a57-a4a8d5157bc3 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:04.431987 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c6416f-9f86-468b-af2b-56b74c2925c1 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:04.465700 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:04.477735 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f9e7ff-f7d9-4951-a72f-7bb01e26eb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:04.480302 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f9e7ff-f7d9-4951-a72f-7bb01e26eb96 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:04.480302 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f9e7ff-f7d9-4951-a72f-7bb01e26eb96 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:04.480302 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f9e7ff-f7d9-4951-a72f-7bb01e26eb96 None admin] PATH_INFO: `/v3/services/network` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:04.481018 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f9e7ff-f7d9-4951-a72f-7bb01e26eb96 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:04.483016 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f9e7ff-f7d9-4951-a72f-7bb01e26eb96 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:04.487802 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfdaca6-fd2b-4fe0-8a57-a4a8d5157bc3 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:04.498924 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-a8f9e7ff-f7d9-4951-a72f-7bb01e26eb96 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 14:32:04.498924 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:04.498924 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:32:04.498924 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:32:04.498924 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:32:04.498924 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:32:04.498924 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:32:04.498924 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:32:04.498924 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 14:32:04.498924 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:32:04.498924 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:32:04.498924 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:32:04.498924 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:04.498924 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:04.498924 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:04.498924 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:04.498924 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:04.499960 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:04.499960 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:04.499960 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:04.499960 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:04.499960 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:04.499960 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:04.499960 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:32:04.499960 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:32:04.499960 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:32:04.499960 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:32:04.499960 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:04.499960 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:04.499960 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:04.499960 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:04.499960 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:04.499960 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:04.500834 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:04.500834 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:04.500834 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:04.500834 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:04.500834 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:04.500834 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:04.500834 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:04.500834 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:04.500834 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:04.500834 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:04.500834 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:32:04.500834 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:32:04.500834 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 14:32:04.500834 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:32:04.500834 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 406/821] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jun 15 14:32:04 2022] GET /identity/v3/services/network => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:04.501630 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f9e7ff-f7d9-4951-a72f-7bb01e26eb96 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:04.532127 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c38deb-b74d-4235-8e37-1a718911e94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:04.533135 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c38deb-b74d-4235-8e37-1a718911e94f None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:04.533592 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c38deb-b74d-4235-8e37-1a718911e94f None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:04.534134 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c38deb-b74d-4235-8e37-1a718911e94f None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:04.535101 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c38deb-b74d-4235-8e37-1a718911e94f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:04.535515 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c38deb-b74d-4235-8e37-1a718911e94f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:04.540634 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c38deb-b74d-4235-8e37-1a718911e94f None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:04.561184 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 416/822] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jun 15 14:32:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:04.562091 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c38deb-b74d-4235-8e37-1a718911e94f None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:04.681092 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c38deb-b74d-4235-8e37-1a718911e94f None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:04.721697 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8cb536-08ea-4902-bc14-d357e2ecfbde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:04.722296 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8cb536-08ea-4902-bc14-d357e2ecfbde None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:04.722574 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8cb536-08ea-4902-bc14-d357e2ecfbde None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:04.722849 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8cb536-08ea-4902-bc14-d357e2ecfbde None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:04.723564 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8cb536-08ea-4902-bc14-d357e2ecfbde None admin] RBAC: Adding query filter params (name=network) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:04.723869 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8cb536-08ea-4902-bc14-d357e2ecfbde None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:04.725288 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8cb536-08ea-4902-bc14-d357e2ecfbde None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:04.731083 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 417/823] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Jun 15 14:32:04 2022] GET /identity/v3/services?name=network => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:04.732117 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8cb536-08ea-4902-bc14-d357e2ecfbde None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:04.741423 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:04.748621 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:04.771467 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f9e7ff-f7d9-4951-a72f-7bb01e26eb96 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:04.808079 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e879280e-3000-4fef-9ac3-ab9b9103dcf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:04.810079 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e879280e-3000-4fef-9ac3-ab9b9103dcf8 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:04.810521 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e879280e-3000-4fef-9ac3-ab9b9103dcf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:04.811198 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e879280e-3000-4fef-9ac3-ab9b9103dcf8 None admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:04.812045 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e879280e-3000-4fef-9ac3-ab9b9103dcf8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:04.812381 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e879280e-3000-4fef-9ac3-ab9b9103dcf8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:04.813819 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e879280e-3000-4fef-9ac3-ab9b9103dcf8 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:04.838069 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 407/824] 10.222.0.42 () {56 vars in 1149 bytes} [Wed Jun 15 14:32:04 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:04.838983 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e879280e-3000-4fef-9ac3-ab9b9103dcf8 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:04.852955 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8cb536-08ea-4902-bc14-d357e2ecfbde None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:04.895647 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89302f19-f9d8-4d3f-a91d-5e0bfeb9a9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:04.896336 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89302f19-f9d8-4d3f-a91d-5e0bfeb9a9f1 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:04.896623 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89302f19-f9d8-4d3f-a91d-5e0bfeb9a9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:04.896900 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89302f19-f9d8-4d3f-a91d-5e0bfeb9a9f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:04.897723 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89302f19-f9d8-4d3f-a91d-5e0bfeb9a9f1 None admin] RBAC: Adding query filter params (role.id=2bd1979f53bb4853a50dd5bee6aa0782, scope.project.id=3094ff8966f149bda784da6a51b1358f, user.id=0927c05c340c48e48b18754888aa9c35) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:04.898033 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89302f19-f9d8-4d3f-a91d-5e0bfeb9a9f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:04.900250 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89302f19-f9d8-4d3f-a91d-5e0bfeb9a9f1 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:04.908958 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 418/825] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jun 15 14:32:04 2022] GET /identity/v3/role_assignments?user.id=0927c05c340c48e48b18754888aa9c35&scope.project.id=3094ff8966f149bda784da6a51b1358f&role.id=2bd1979f53bb4853a50dd5bee6aa0782 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:04.909809 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89302f19-f9d8-4d3f-a91d-5e0bfeb9a9f1 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:04.972579 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e879280e-3000-4fef-9ac3-ab9b9103dcf8 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:05.021218 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:05.061435 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8085ab98-7202-4c24-affc-f08bc0461ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:05.062463 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8085ab98-7202-4c24-affc-f08bc0461ecc None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:05.062894 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8085ab98-7202-4c24-affc-f08bc0461ecc None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:05.063348 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8085ab98-7202-4c24-affc-f08bc0461ecc None admin] PATH_INFO: `/v3/services` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:05.064266 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8085ab98-7202-4c24-affc-f08bc0461ecc None admin] RBAC: Adding query filter params (type=network) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:05.064738 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8085ab98-7202-4c24-affc-f08bc0461ecc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:05.066273 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8085ab98-7202-4c24-affc-f08bc0461ecc None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:05.070665 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:05.086772 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 408/826] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Jun 15 14:32:04 2022] GET /identity/v3/services?type=network => generated 350 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:05.095447 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8085ab98-7202-4c24-affc-f08bc0461ecc None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:05.102311 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89302f19-f9d8-4d3f-a91d-5e0bfeb9a9f1 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:05.165865 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e450e664-ebc1-4dce-bc5f-ef11fab4c3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:05.166915 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450e664-ebc1-4dce-bc5f-ef11fab4c3a2 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:05.167350 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450e664-ebc1-4dce-bc5f-ef11fab4c3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:05.168346 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450e664-ebc1-4dce-bc5f-ef11fab4c3a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:05.184040 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e450e664-ebc1-4dce-bc5f-ef11fab4c3a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:05.186158 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e450e664-ebc1-4dce-bc5f-ef11fab4c3a2 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:05.209032 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-e450e664-ebc1-4dce-bc5f-ef11fab4c3a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:05.209032 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:05.209032 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:05.209032 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:05.209032 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:05.209032 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:05.209032 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:05.209032 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:05.209032 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:05.209032 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:05.209032 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:05.209032 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:05.209032 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:32:05.209032 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:32:05.209032 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:32:05.210218 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:32:05.210218 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:05.210218 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:05.210218 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:05.210218 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:05.210218 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:05.210218 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:05.210218 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:05.210218 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:05.210218 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:05.210218 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:05.210218 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:05.210218 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:05.210218 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:05.210218 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:05.210218 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:05.210218 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:05.211197 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:32:05.211197 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:32:05.211197 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:05.211197 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:05.213346 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 419/827] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:32:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:05.221884 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8085ab98-7202-4c24-affc-f08bc0461ecc None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:05.222061 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e450e664-ebc1-4dce-bc5f-ef11fab4c3a2 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:05.262367 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f216ece-bcb8-4880-89c7-1efb7072afe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:05.263024 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f216ece-bcb8-4880-89c7-1efb7072afe3 None admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:05.263352 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f216ece-bcb8-4880-89c7-1efb7072afe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:05.263576 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f216ece-bcb8-4880-89c7-1efb7072afe3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:05.264351 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f216ece-bcb8-4880-89c7-1efb7072afe3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:05.265707 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f216ece-bcb8-4880-89c7-1efb7072afe3 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:05.337531 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 409/828] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Jun 15 14:32:05 2022] POST /identity/v3/endpoints => generated 327 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:32:05.358935 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e450e664-ebc1-4dce-bc5f-ef11fab4c3a2 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:05.386406 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:05.415614 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139ad672-ec67-4191-a08b-820c9113e434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:05.416326 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139ad672-ec67-4191-a08b-820c9113e434 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:05.416613 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139ad672-ec67-4191-a08b-820c9113e434 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:05.416890 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139ad672-ec67-4191-a08b-820c9113e434 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:05.417733 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139ad672-ec67-4191-a08b-820c9113e434 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:05.418628 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139ad672-ec67-4191-a08b-820c9113e434 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:05.420192 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139ad672-ec67-4191-a08b-820c9113e434 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:05.440664 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 420/829] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:32:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:05.446394 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f216ece-bcb8-4880-89c7-1efb7072afe3 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:05.519273 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:05.573259 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f216ece-bcb8-4880-89c7-1efb7072afe3 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:05.621708 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db5fdd8-b168-4615-8aae-218da1f02f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:05.622886 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db5fdd8-b168-4615-8aae-218da1f02f44 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:05.623370 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db5fdd8-b168-4615-8aae-218da1f02f44 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:05.623849 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db5fdd8-b168-4615-8aae-218da1f02f44 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:05.631445 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db5fdd8-b168-4615-8aae-218da1f02f44 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:05.633189 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db5fdd8-b168-4615-8aae-218da1f02f44 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:05.648574 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-2db5fdd8-b168-4615-8aae-218da1f02f44 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:32:05.648574 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:05.648574 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:05.648574 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:05.648574 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:05.648574 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:05.648574 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:05.648574 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:05.648574 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:05.648574 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:05.648574 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:05.648574 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:05.648574 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:32:05.648574 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:32:05.648574 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:32:05.649755 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:32:05.649755 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:05.649755 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:05.649755 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:05.649755 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:05.649755 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:05.649755 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:05.649755 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:05.649755 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:05.649755 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:05.649755 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:05.649755 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:05.649755 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:05.649755 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:05.649755 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:05.649755 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:05.649755 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:05.650723 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:32:05.650723 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:32:05.650723 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:32:05.650723 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:32:05.650723 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:32:05.650723 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:32:05.650723 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:32:05.650723 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:32:05.653526 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 410/830] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jun 15 14:32:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:05.658829 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139ad672-ec67-4191-a08b-820c9113e434 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:05.775468 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139ad672-ec67-4191-a08b-820c9113e434 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:05.821662 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d835e4-8740-4852-8891-28eccab03a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:05.822325 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d835e4-8740-4852-8891-28eccab03a06 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:05.822604 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d835e4-8740-4852-8891-28eccab03a06 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:05.822877 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d835e4-8740-4852-8891-28eccab03a06 None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:05.823667 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d835e4-8740-4852-8891-28eccab03a06 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:05.824581 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d835e4-8740-4852-8891-28eccab03a06 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:05.826106 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d835e4-8740-4852-8891-28eccab03a06 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:05.841260 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 421/831] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jun 15 14:32:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:05.846814 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db5fdd8-b168-4615-8aae-218da1f02f44 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:05.960961 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e0f153-b865-47ce-9e2f-e1879bb7af96 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:05.961474 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e0f153-b865-47ce-9e2f-e1879bb7af96 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:05.961969 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e0f153-b865-47ce-9e2f-e1879bb7af96 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:05.964564 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 422/832] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:32:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:32:05.969213 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db5fdd8-b168-4615-8aae-218da1f02f44 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:05.970881 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e86d25-2fe3-4d15-a598-85c89da1983a None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:05.973373 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e86d25-2fe3-4d15-a598-85c89da1983a None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:05.973373 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e86d25-2fe3-4d15-a598-85c89da1983a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:05.985608 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:05.989078 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-93e86d25-2fe3-4d15-a598-85c89da1983a None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:32:05.990308 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-93e86d25-2fe3-4d15-a598-85c89da1983a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:32:06.005000 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e427d8-8b3e-4a36-96f8-5f4b3d9243c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:06.006107 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e427d8-8b3e-4a36-96f8-5f4b3d9243c5 None admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:06.006452 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e427d8-8b3e-4a36-96f8-5f4b3d9243c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:06.006841 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e427d8-8b3e-4a36-96f8-5f4b3d9243c5 None admin] PATH_INFO: `/v3/projects/3094ff8966f149bda784da6a51b1358f/users/19d82c04f9994654adfb10f4bd0573c2/roles/305a1ca2e6f740eba7dc696849e933eb` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:06.023662 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e427d8-8b3e-4a36-96f8-5f4b3d9243c5 None admin] RBAC: Authorizing `identity:create_grant(project_id=3094ff8966f149bda784da6a51b1358f, user_id=19d82c04f9994654adfb10f4bd0573c2, role_id=305a1ca2e6f740eba7dc696849e933eb)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:06.024510 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:06.026626 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e427d8-8b3e-4a36-96f8-5f4b3d9243c5 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:06.066476 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 423/833] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:32:05 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:32:06.079921 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 411/834] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Jun 15 14:32:05 2022] PUT /identity/v3/projects/3094ff8966f149bda784da6a51b1358f/users/19d82c04f9994654adfb10f4bd0573c2/roles/305a1ca2e6f740eba7dc696849e933eb => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:32:06.091757 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e86d25-2fe3-4d15-a598-85c89da1983a None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:06.205697 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e86d25-2fe3-4d15-a598-85c89da1983a None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:06.244023 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af88b407-6ed8-4f02-8144-e0b5b3a9c8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:06.244894 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af88b407-6ed8-4f02-8144-e0b5b3a9c8bd None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:06.245178 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af88b407-6ed8-4f02-8144-e0b5b3a9c8bd None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:06.245456 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af88b407-6ed8-4f02-8144-e0b5b3a9c8bd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:06.259228 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af88b407-6ed8-4f02-8144-e0b5b3a9c8bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:32:06.267881 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-af88b407-6ed8-4f02-8144-e0b5b3a9c8bd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:32:06.267881 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:06.267881 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:06.267881 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:06.267881 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:06.267881 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:06.267881 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:06.267881 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:06.267881 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:06.267881 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:06.267881 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:06.267881 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:06.267881 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:32:06.267881 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:32:06.267881 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:32:06.267881 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:32:06.268938 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:32:06.268938 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:32:06.268938 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:32:06.268938 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:32:06.268938 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:06.268938 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:06.268938 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:06.268938 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:06.268938 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:06.268938 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:06.268938 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:06.268938 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:06.268938 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:06.268938 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:06.268938 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:06.268938 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:06.270717 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:06.270717 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:06.270717 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:06.270717 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:06.270717 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:32:06.270717 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:32:06.270717 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:32:06.270717 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:32:06.270717 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:32:06.270717 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:32:06.270717 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:32:06.270717 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:06.270717 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 424/835] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jun 15 14:32:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:06.275964 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e427d8-8b3e-4a36-96f8-5f4b3d9243c5 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:06.319815 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a84d01-ae69-466b-8c36-25f4b9dd06a5 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:06.320023 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a84d01-ae69-466b-8c36-25f4b9dd06a5 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:06.320393 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a84d01-ae69-466b-8c36-25f4b9dd06a5 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:06.322143 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 425/836] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:32:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:32:06.328834 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83776a03-f1ea-4b6a-9e8e-373f1bae3d24 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:06.329119 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83776a03-f1ea-4b6a-9e8e-373f1bae3d24 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:06.329397 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83776a03-f1ea-4b6a-9e8e-373f1bae3d24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:06.346974 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-83776a03-f1ea-4b6a-9e8e-373f1bae3d24 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:32:06.348176 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-83776a03-f1ea-4b6a-9e8e-373f1bae3d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:32:06.369724 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:06.397951 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e427d8-8b3e-4a36-96f8-5f4b3d9243c5 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:06.436502 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a733643-cc0c-4a24-a88f-5920d5c9e9e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:06.437218 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a733643-cc0c-4a24-a88f-5920d5c9e9e4 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:06.437880 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a733643-cc0c-4a24-a88f-5920d5c9e9e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:06.437880 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a733643-cc0c-4a24-a88f-5920d5c9e9e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:06.438602 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a733643-cc0c-4a24-a88f-5920d5c9e9e4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:06.438914 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a733643-cc0c-4a24-a88f-5920d5c9e9e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:06.440418 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a733643-cc0c-4a24-a88f-5920d5c9e9e4 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:06.444694 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 426/837] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:32:06 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (3 switches on core 0) Jun 15 14:32:06.456179 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 412/838] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jun 15 14:32:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:06.461866 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83776a03-f1ea-4b6a-9e8e-373f1bae3d24 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:06.469907 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a733643-cc0c-4a24-a88f-5920d5c9e9e4 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:06.503716 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:06.592401 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83776a03-f1ea-4b6a-9e8e-373f1bae3d24 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:06.598817 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a733643-cc0c-4a24-a88f-5920d5c9e9e4 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:06.627685 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2c4adc-e4db-4b06-b112-9c0c384d4a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:06.628513 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c4adc-e4db-4b06-b112-9c0c384d4a3d None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:06.628796 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c4adc-e4db-4b06-b112-9c0c384d4a3d None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:06.629073 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c4adc-e4db-4b06-b112-9c0c384d4a3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:06.639446 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2c4adc-e4db-4b06-b112-9c0c384d4a3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:06.640931 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2c4adc-e4db-4b06-b112-9c0c384d4a3d None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:06.644296 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb83ea8-b085-448d-b17f-ed74ed00d652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:06.645371 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb83ea8-b085-448d-b17f-ed74ed00d652 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:06.645866 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb83ea8-b085-448d-b17f-ed74ed00d652 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:06.646338 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb83ea8-b085-448d-b17f-ed74ed00d652 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:06.647190 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb83ea8-b085-448d-b17f-ed74ed00d652 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:06.650942 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb83ea8-b085-448d-b17f-ed74ed00d652 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:06.660478 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-7a2c4adc-e4db-4b06-b112-9c0c384d4a3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:06.660478 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:06.660478 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:06.660478 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:06.660478 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:06.660478 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:06.660478 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:06.660478 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:06.660478 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:06.660478 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:06.660478 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:06.660478 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:06.660478 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:32:06.660478 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:32:06.660478 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:32:06.661375 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:32:06.661375 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:06.661375 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:06.661375 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:06.661375 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:06.661375 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:06.661375 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:06.661375 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:06.661375 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:06.661375 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:06.661375 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:06.661375 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:06.661375 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:06.661375 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:06.661375 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:06.661375 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:06.661375 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:06.662292 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:32:06.662292 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:32:06.662292 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:06.662292 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:06.666339 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 427/839] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:32:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:06.667241 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-bfb83ea8-b085-448d-b17f-ed74ed00d652 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 14:32:06.667241 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:06.667241 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:32:06.667241 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:32:06.667241 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:32:06.667241 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:32:06.667241 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:32:06.667241 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:32:06.667241 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 14:32:06.667241 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:32:06.667241 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:32:06.667241 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:32:06.667241 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:06.667241 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:06.667241 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:06.667241 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:06.667241 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:06.668826 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:06.668826 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:06.668826 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:06.668826 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:06.668826 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:06.668826 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:06.668826 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:32:06.668826 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:32:06.668826 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:32:06.668826 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:32:06.668826 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:06.668826 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:06.668826 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:06.668826 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:06.668826 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:06.668826 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:06.669791 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:06.669791 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:06.669791 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:06.669791 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:06.669791 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:06.669791 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:06.669791 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:06.669791 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:06.669791 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:06.669791 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:06.669791 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:32:06.669791 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:32:06.669791 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 14:32:06.669791 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:32:06.670562 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2c4adc-e4db-4b06-b112-9c0c384d4a3d None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:06.672431 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 413/840] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Jun 15 14:32:06 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:32:06.684627 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb83ea8-b085-448d-b17f-ed74ed00d652 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:06.687091 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:06.773133 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:06.781846 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb83ea8-b085-448d-b17f-ed74ed00d652 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:06.786577 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2c4adc-e4db-4b06-b112-9c0c384d4a3d None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:06.815526 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75067202-6274-45fc-90f6-7d57c20500bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:06.816214 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75067202-6274-45fc-90f6-7d57c20500bf None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:06.816505 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75067202-6274-45fc-90f6-7d57c20500bf None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:06.816766 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75067202-6274-45fc-90f6-7d57c20500bf None admin] PATH_INFO: `/v3/services` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:06.817473 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75067202-6274-45fc-90f6-7d57c20500bf None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:06.817758 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75067202-6274-45fc-90f6-7d57c20500bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:06.819103 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75067202-6274-45fc-90f6-7d57c20500bf None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:06.828018 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 414/841] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Jun 15 14:32:06 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:06.830360 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79da911-070f-4507-8ee0-500cd8e5e032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:06.831030 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79da911-070f-4507-8ee0-500cd8e5e032 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:06.831313 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79da911-070f-4507-8ee0-500cd8e5e032 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:06.831590 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79da911-070f-4507-8ee0-500cd8e5e032 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:06.832383 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79da911-070f-4507-8ee0-500cd8e5e032 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:06.832690 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79da911-070f-4507-8ee0-500cd8e5e032 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:06.833393 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75067202-6274-45fc-90f6-7d57c20500bf None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:06.834044 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79da911-070f-4507-8ee0-500cd8e5e032 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:06.852019 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 428/842] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:32:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:06.853496 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79da911-070f-4507-8ee0-500cd8e5e032 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:06.928911 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:06.937260 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75067202-6274-45fc-90f6-7d57c20500bf None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:06.977219 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0154fcf1-bc87-44d8-9c5e-763c88a49fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:06.978287 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0154fcf1-bc87-44d8-9c5e-763c88a49fc3 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:06.978802 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0154fcf1-bc87-44d8-9c5e-763c88a49fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:06.979270 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0154fcf1-bc87-44d8-9c5e-763c88a49fc3 None admin] PATH_INFO: `/v3/services` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:06.980910 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0154fcf1-bc87-44d8-9c5e-763c88a49fc3 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:06.981508 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0154fcf1-bc87-44d8-9c5e-763c88a49fc3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:06.983188 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0154fcf1-bc87-44d8-9c5e-763c88a49fc3 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:06.996721 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 415/843] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Jun 15 14:32:06 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:07.094093 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79da911-070f-4507-8ee0-500cd8e5e032 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:07.132642 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c58532-387d-4601-aa89-8087ece3067a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:07.133365 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c58532-387d-4601-aa89-8087ece3067a None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:07.133648 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c58532-387d-4601-aa89-8087ece3067a None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:07.133922 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c58532-387d-4601-aa89-8087ece3067a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:07.142232 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c58532-387d-4601-aa89-8087ece3067a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:07.143685 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c58532-387d-4601-aa89-8087ece3067a None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:07.160208 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-c8c58532-387d-4601-aa89-8087ece3067a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 15 14:32:07.160208 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:07.160208 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:07.160208 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:07.160208 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:07.160208 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:07.160208 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:07.160208 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:07.160208 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:07.160208 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:07.160208 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:07.160208 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:07.160208 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:32:07.160208 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:32:07.160208 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:32:07.160208 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:32:07.161201 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:07.161201 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:07.161201 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:32:07.161201 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:32:07.161201 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:32:07.161201 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:32:07.161201 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:07.161201 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:07.161201 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:07.161201 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:07.161201 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:07.161201 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:07.161201 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:07.161201 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:07.161201 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:07.161201 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:07.161201 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:07.161201 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:07.162139 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:07.162139 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:07.162139 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:32:07.162139 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:32:07.162139 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:32:07.162139 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:32:07.162139 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:32:07.162139 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:32:07.162139 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 15 14:32:07.162139 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:07.166227 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 429/844] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jun 15 14:32:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:07.167619 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0154fcf1-bc87-44d8-9c5e-763c88a49fc3 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:07.195081 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:07.277535 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0154fcf1-bc87-44d8-9c5e-763c88a49fc3 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:07.313052 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f5baeb-eb23-4450-b93a-529702a39f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:07.313718 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5baeb-eb23-4450-b93a-529702a39f25 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:07.313999 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5baeb-eb23-4450-b93a-529702a39f25 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:07.314273 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5baeb-eb23-4450-b93a-529702a39f25 None admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:07.315100 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f5baeb-eb23-4450-b93a-529702a39f25 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:07.315407 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f5baeb-eb23-4450-b93a-529702a39f25 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:07.316844 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f5baeb-eb23-4450-b93a-529702a39f25 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:07.336117 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 416/845] 10.222.0.42 () {56 vars in 1135 bytes} [Wed Jun 15 14:32:07 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:07.342326 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c58532-387d-4601-aa89-8087ece3067a None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:07.449839 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c58532-387d-4601-aa89-8087ece3067a None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:07.480652 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:07.490720 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7050776f-6e8b-44ce-a5c7-a2d2c9c44beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:07.492280 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7050776f-6e8b-44ce-a5c7-a2d2c9c44beb None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:07.492280 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7050776f-6e8b-44ce-a5c7-a2d2c9c44beb None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:07.492745 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7050776f-6e8b-44ce-a5c7-a2d2c9c44beb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:07.540580 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bcd620-4a6a-4990-8bf9-1ca2ac3b3b32 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:07.540580 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bcd620-4a6a-4990-8bf9-1ca2ac3b3b32 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:07.540580 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bcd620-4a6a-4990-8bf9-1ca2ac3b3b32 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:07.540580 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 417/846] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:32:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:32:07.546372 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671569d2-a03a-4f44-81e8-17691a36945c None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:07.546660 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671569d2-a03a-4f44-81e8-17691a36945c None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:07.546937 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671569d2-a03a-4f44-81e8-17691a36945c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:07.564889 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-671569d2-a03a-4f44-81e8-17691a36945c None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:32:07.566118 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-671569d2-a03a-4f44-81e8-17691a36945c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:32:07.580906 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7050776f-6e8b-44ce-a5c7-a2d2c9c44beb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:07.582877 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7050776f-6e8b-44ce-a5c7-a2d2c9c44beb None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:07.657175 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 418/847] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:32:07 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:32:07.680039 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671569d2-a03a-4f44-81e8-17691a36945c None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:07.690709 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:07.712963 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 430/848] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jun 15 14:32:07 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:32:07.720776 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7050776f-6e8b-44ce-a5c7-a2d2c9c44beb None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:07.747739 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:07.799085 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671569d2-a03a-4f44-81e8-17691a36945c None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:07.822490 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7050776f-6e8b-44ce-a5c7-a2d2c9c44beb None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:07.841352 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01cf648-9233-4d64-99fd-3467e0b10184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:07.842308 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01cf648-9233-4d64-99fd-3467e0b10184 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:07.842879 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01cf648-9233-4d64-99fd-3467e0b10184 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:07.843154 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01cf648-9233-4d64-99fd-3467e0b10184 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:07.843808 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01cf648-9233-4d64-99fd-3467e0b10184 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:07.845288 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01cf648-9233-4d64-99fd-3467e0b10184 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:07.860867 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cfbd8f-c9bb-4b40-9c01-db8def1adb01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:07.861136 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-e01cf648-9233-4d64-99fd-3467e0b10184 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 14:32:07.861136 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:07.861136 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:32:07.861136 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:32:07.861136 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:32:07.861136 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:32:07.861136 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:32:07.861136 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:32:07.861136 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 14:32:07.861136 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:32:07.861136 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:32:07.861136 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:32:07.861136 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:07.861136 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:07.861136 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:07.861136 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:07.862035 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:07.862035 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:07.862035 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:07.862035 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:07.862035 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:07.862035 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:07.862035 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:07.862035 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:32:07.862035 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:32:07.862035 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:32:07.862035 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:32:07.862035 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:07.862035 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:07.862035 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:07.862035 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:07.862035 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:07.862894 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:07.862894 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:07.862894 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:07.862894 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:07.862894 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:07.862894 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:07.862894 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:07.862894 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:07.862894 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:07.862894 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:07.862894 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:07.862894 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:32:07.862894 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:32:07.862894 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 14:32:07.862894 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:32:07.863680 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cfbd8f-c9bb-4b40-9c01-db8def1adb01 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:07.863680 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cfbd8f-c9bb-4b40-9c01-db8def1adb01 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:07.863680 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cfbd8f-c9bb-4b40-9c01-db8def1adb01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:07.863867 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 419/849] 10.222.0.42 () {56 vars in 1132 bytes} [Wed Jun 15 14:32:07 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:07.869830 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01cf648-9233-4d64-99fd-3467e0b10184 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:07.874841 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cfbd8f-c9bb-4b40-9c01-db8def1adb01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:07.876655 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cfbd8f-c9bb-4b40-9c01-db8def1adb01 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:07.895623 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-52cfbd8f-c9bb-4b40-9c01-db8def1adb01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:07.895623 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:07.895623 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:07.895623 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:07.895623 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:07.895623 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:07.895623 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:07.895623 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:07.895623 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:07.895623 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:07.895623 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:07.895623 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:07.895623 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:32:07.895623 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:32:07.895623 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:32:07.896600 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:32:07.896600 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:07.896600 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:07.896600 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:07.896600 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:07.896600 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:07.896600 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:07.896600 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:07.896600 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:07.896600 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:07.896600 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:07.896600 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:07.896600 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:07.896600 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:07.896600 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:07.896600 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:07.896600 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:07.897504 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:32:07.897504 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:32:07.897504 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:07.897504 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:07.905221 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 431/850] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:32:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:07.906751 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cfbd8f-c9bb-4b40-9c01-db8def1adb01 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:07.950397 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:07.984401 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01cf648-9233-4d64-99fd-3467e0b10184 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:08.017654 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:08.024473 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a90e4c-1014-4aa8-b95b-d54f77dc2a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:08.025458 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a90e4c-1014-4aa8-b95b-d54f77dc2a71 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:08.026020 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a90e4c-1014-4aa8-b95b-d54f77dc2a71 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:08.026495 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a90e4c-1014-4aa8-b95b-d54f77dc2a71 None admin] PATH_INFO: `/v3/services` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:08.027497 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a90e4c-1014-4aa8-b95b-d54f77dc2a71 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:08.028305 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a90e4c-1014-4aa8-b95b-d54f77dc2a71 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:08.029879 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a90e4c-1014-4aa8-b95b-d54f77dc2a71 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:08.035630 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 420/851] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Jun 15 14:32:07 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:08.040559 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cfbd8f-c9bb-4b40-9c01-db8def1adb01 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:08.042163 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a90e4c-1014-4aa8-b95b-d54f77dc2a71 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:08.079546 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2caf6c0-e7ae-49f7-90a4-4e4ecb158855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:08.080198 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2caf6c0-e7ae-49f7-90a4-4e4ecb158855 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:08.080483 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2caf6c0-e7ae-49f7-90a4-4e4ecb158855 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:08.080756 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2caf6c0-e7ae-49f7-90a4-4e4ecb158855 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:08.081484 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2caf6c0-e7ae-49f7-90a4-4e4ecb158855 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:08.081789 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2caf6c0-e7ae-49f7-90a4-4e4ecb158855 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:08.083270 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2caf6c0-e7ae-49f7-90a4-4e4ecb158855 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:08.097007 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 432/852] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:32:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:32:08.103706 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2caf6c0-e7ae-49f7-90a4-4e4ecb158855 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:08.175793 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a90e4c-1014-4aa8-b95b-d54f77dc2a71 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:08.219761 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67107f60-9974-4ef6-a504-f676aa3a317e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:08.220485 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67107f60-9974-4ef6-a504-f676aa3a317e None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:08.220724 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67107f60-9974-4ef6-a504-f676aa3a317e None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:08.221000 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67107f60-9974-4ef6-a504-f676aa3a317e None admin] PATH_INFO: `/v3/services` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:08.221716 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67107f60-9974-4ef6-a504-f676aa3a317e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:08.222011 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67107f60-9974-4ef6-a504-f676aa3a317e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:08.223370 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67107f60-9974-4ef6-a504-f676aa3a317e None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:08.225811 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:08.229554 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2caf6c0-e7ae-49f7-90a4-4e4ecb158855 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:08.236496 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 421/853] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Jun 15 14:32:08 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:32:08.274407 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e451c070-0b81-4f94-b012-7caabb0c22fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:08.275648 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e451c070-0b81-4f94-b012-7caabb0c22fe None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:08.276163 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e451c070-0b81-4f94-b012-7caabb0c22fe None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:08.276663 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e451c070-0b81-4f94-b012-7caabb0c22fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:08.285870 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e451c070-0b81-4f94-b012-7caabb0c22fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:08.287800 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e451c070-0b81-4f94-b012-7caabb0c22fe None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:08.291445 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:08.305317 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-e451c070-0b81-4f94-b012-7caabb0c22fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:32:08.305317 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:08.305317 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:08.305317 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:08.305317 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:08.305317 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:08.305317 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:08.305317 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:08.305317 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:08.305317 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:08.305317 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:08.305317 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:08.305317 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:32:08.305317 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:32:08.305317 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:32:08.306985 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:32:08.306985 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:08.306985 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:08.306985 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:08.306985 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:08.306985 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:08.306985 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:08.306985 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:08.306985 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:08.306985 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:08.306985 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:08.306985 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:08.306985 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:08.306985 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:08.306985 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:08.306985 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:08.306985 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:08.307867 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:32:08.307867 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:32:08.307867 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:32:08.307867 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:32:08.307867 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:32:08.307867 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:32:08.307867 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:32:08.307867 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:08.314749 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 433/854] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jun 15 14:32:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:08.316517 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67107f60-9974-4ef6-a504-f676aa3a317e None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:08.338960 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adeb23a-1489-4874-994c-e3f8de7859f4 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:08.339303 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adeb23a-1489-4874-994c-e3f8de7859f4 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:08.340348 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adeb23a-1489-4874-994c-e3f8de7859f4 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:08.343257 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 434/855] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:32:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:32:08.348815 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04218cf4-6a99-4fc5-91ca-52fb7366b318 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:08.349239 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04218cf4-6a99-4fc5-91ca-52fb7366b318 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:08.349617 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04218cf4-6a99-4fc5-91ca-52fb7366b318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:08.368139 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-04218cf4-6a99-4fc5-91ca-52fb7366b318 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:32:08.369206 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-04218cf4-6a99-4fc5-91ca-52fb7366b318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:32:08.440789 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67107f60-9974-4ef6-a504-f676aa3a317e None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:08.452202 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 435/856] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:32:08 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:32:08.470936 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04218cf4-6a99-4fc5-91ca-52fb7366b318 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:08.476621 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9748d3a-bae2-44cf-b61b-4afc100219fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:08.477283 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9748d3a-bae2-44cf-b61b-4afc100219fa None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:08.477563 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9748d3a-bae2-44cf-b61b-4afc100219fa None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:08.477838 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9748d3a-bae2-44cf-b61b-4afc100219fa None admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:08.478620 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9748d3a-bae2-44cf-b61b-4afc100219fa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:08.478929 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9748d3a-bae2-44cf-b61b-4afc100219fa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:08.480438 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9748d3a-bae2-44cf-b61b-4afc100219fa None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:08.494094 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 422/857] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jun 15 14:32:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:08.501357 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9748d3a-bae2-44cf-b61b-4afc100219fa None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:08.580753 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:08.610893 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04218cf4-6a99-4fc5-91ca-52fb7366b318 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:08.631266 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9748d3a-bae2-44cf-b61b-4afc100219fa None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:08.643748 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:08.649839 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083849a9-2cb3-4669-8107-dca5def1b7df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:08.650846 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083849a9-2cb3-4669-8107-dca5def1b7df None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:08.651372 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083849a9-2cb3-4669-8107-dca5def1b7df None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:08.651794 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083849a9-2cb3-4669-8107-dca5def1b7df None admin] PATH_INFO: `/v3/roles/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:08.663198 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083849a9-2cb3-4669-8107-dca5def1b7df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:32:08.671793 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-083849a9-2cb3-4669-8107-dca5def1b7df None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:32:08.671793 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:08.671793 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:08.671793 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:08.671793 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:08.671793 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:08.671793 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:08.671793 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:08.671793 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:08.671793 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:08.671793 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:08.671793 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:08.671793 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:32:08.671793 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:32:08.671793 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:32:08.671793 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:32:08.672904 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:32:08.672904 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:32:08.672904 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:32:08.672904 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:32:08.672904 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:08.672904 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:08.672904 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:08.672904 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:08.672904 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:08.672904 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:08.672904 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:08.672904 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:08.672904 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:08.672904 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:08.672904 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:08.672904 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:08.673792 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:08.673792 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:08.673792 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:08.673792 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:08.673792 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:32:08.673792 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:32:08.673792 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:32:08.673792 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:32:08.673792 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:32:08.673792 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:32:08.673792 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:32:08.673792 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:08.675934 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3e1e47-aed5-4ea0-80d1-e3ae5ddaf921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:08.676673 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e1e47-aed5-4ea0-80d1-e3ae5ddaf921 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:08.677015 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 436/858] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Jun 15 14:32:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:32:08.681453 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e1e47-aed5-4ea0-80d1-e3ae5ddaf921 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:08.681750 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e1e47-aed5-4ea0-80d1-e3ae5ddaf921 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:08.682594 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3e1e47-aed5-4ea0-80d1-e3ae5ddaf921 None admin] RBAC: Adding query filter params (role.id=2bd1979f53bb4853a50dd5bee6aa0782, scope.project.id=3094ff8966f149bda784da6a51b1358f, user.id=c04dc5832333498e96a7e02bbfb9484c) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:08.682914 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3e1e47-aed5-4ea0-80d1-e3ae5ddaf921 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:08.684914 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3e1e47-aed5-4ea0-80d1-e3ae5ddaf921 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:08.685724 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083849a9-2cb3-4669-8107-dca5def1b7df None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:08.696179 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 423/859] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jun 15 14:32:08 2022] GET /identity/v3/role_assignments?user.id=c04dc5832333498e96a7e02bbfb9484c&scope.project.id=3094ff8966f149bda784da6a51b1358f&role.id=2bd1979f53bb4853a50dd5bee6aa0782 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:08.802883 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083849a9-2cb3-4669-8107-dca5def1b7df None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:08.840996 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5c44fb-a608-4916-a888-1b248bbc1e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:08.841827 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5c44fb-a608-4916-a888-1b248bbc1e34 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:08.842111 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5c44fb-a608-4916-a888-1b248bbc1e34 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:08.842389 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5c44fb-a608-4916-a888-1b248bbc1e34 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:08.843149 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5c44fb-a608-4916-a888-1b248bbc1e34 None admin] RBAC: Adding query filter params (name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:08.843454 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5c44fb-a608-4916-a888-1b248bbc1e34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:08.844974 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5c44fb-a608-4916-a888-1b248bbc1e34 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:08.854532 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 437/860] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Jun 15 14:32:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:32:08.861166 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3e1e47-aed5-4ea0-80d1-e3ae5ddaf921 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:08.969941 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3e1e47-aed5-4ea0-80d1-e3ae5ddaf921 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:08.994526 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:09.006115 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048dcbb3-0acc-4b7c-990f-5ea8ab7fde20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:09.006903 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048dcbb3-0acc-4b7c-990f-5ea8ab7fde20 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:09.007177 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048dcbb3-0acc-4b7c-990f-5ea8ab7fde20 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:09.007449 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048dcbb3-0acc-4b7c-990f-5ea8ab7fde20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:09.017834 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048dcbb3-0acc-4b7c-990f-5ea8ab7fde20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:09.019265 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048dcbb3-0acc-4b7c-990f-5ea8ab7fde20 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:09.037044 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-048dcbb3-0acc-4b7c-990f-5ea8ab7fde20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:09.037044 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:09.037044 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:09.037044 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:09.037044 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:09.037044 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:09.037044 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:09.037044 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:09.037044 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:09.037044 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:09.037044 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:09.037044 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:09.037044 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:32:09.037044 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:32:09.037044 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:32:09.039153 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:32:09.039153 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:09.039153 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:09.039153 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:09.039153 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:09.039153 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:09.039153 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:09.039153 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:09.039153 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:09.039153 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:09.039153 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:09.039153 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:09.039153 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:09.039153 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:09.039153 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:09.039153 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:09.039153 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:09.041314 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:32:09.041314 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:32:09.041314 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:09.041314 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:32:09.041314 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 424/861] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:32:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:09.044745 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5c44fb-a608-4916-a888-1b248bbc1e34 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:09.101887 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:09.154849 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5c44fb-a608-4916-a888-1b248bbc1e34 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:09.191406 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f8a07c-c361-4c60-8ff6-2dd76ddac87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:09.192121 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f8a07c-c361-4c60-8ff6-2dd76ddac87a None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:09.192418 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f8a07c-c361-4c60-8ff6-2dd76ddac87a None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:09.192703 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f8a07c-c361-4c60-8ff6-2dd76ddac87a None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:09.193468 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f8a07c-c361-4c60-8ff6-2dd76ddac87a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:09.193783 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f8a07c-c361-4c60-8ff6-2dd76ddac87a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:09.195220 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f8a07c-c361-4c60-8ff6-2dd76ddac87a None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:09.211192 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 438/862] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:32:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:32:09.216603 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048dcbb3-0acc-4b7c-990f-5ea8ab7fde20 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:09.259528 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ef7e75-1c57-4d98-861b-c5e1803061d4 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:09.259818 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ef7e75-1c57-4d98-861b-c5e1803061d4 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:09.262232 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ef7e75-1c57-4d98-861b-c5e1803061d4 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:09.262232 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 439/863] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:32:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:32:09.268854 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b301d-e954-4072-a88f-9759274142c8 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:09.269245 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b301d-e954-4072-a88f-9759274142c8 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:09.269529 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b301d-e954-4072-a88f-9759274142c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:09.290916 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-a09b301d-e954-4072-a88f-9759274142c8 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:32:09.292428 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a09b301d-e954-4072-a88f-9759274142c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:32:09.333651 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048dcbb3-0acc-4b7c-990f-5ea8ab7fde20 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:09.373140 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 440/864] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:32:09 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:32:09.384005 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67d5291-083d-45c4-814e-0f8f68fd2590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:09.385044 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67d5291-083d-45c4-814e-0f8f68fd2590 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:09.385479 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67d5291-083d-45c4-814e-0f8f68fd2590 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:09.385907 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67d5291-083d-45c4-814e-0f8f68fd2590 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:09.394147 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67d5291-083d-45c4-814e-0f8f68fd2590 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:09.395875 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67d5291-083d-45c4-814e-0f8f68fd2590 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:09.398132 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09b301d-e954-4072-a88f-9759274142c8 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:09.424465 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-c67d5291-083d-45c4-814e-0f8f68fd2590 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 14:32:09.424465 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:09.424465 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:09.424465 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:09.424465 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:09.424465 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:09.424465 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:09.424465 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:09.424465 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:09.424465 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:09.424465 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:09.424465 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:09.424465 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:32:09.424465 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:32:09.424465 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:32:09.424465 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:32:09.425470 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:09.425470 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:09.425470 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:32:09.425470 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:32:09.425470 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:32:09.425470 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:32:09.425470 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:09.425470 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:09.425470 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:09.425470 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:09.425470 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:09.425470 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:09.425470 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:09.425470 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:09.425470 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:09.425470 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:09.425470 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:09.425470 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:09.426401 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:09.426401 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:09.426401 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:32:09.426401 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:32:09.426401 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:32:09.426401 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:32:09.426401 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:32:09.426401 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:32:09.426401 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 14:32:09.426401 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:32:09.429541 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 425/865] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Jun 15 14:32:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:09.434606 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67d5291-083d-45c4-814e-0f8f68fd2590 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:09.454616 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:09.484123 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:09.518625 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09b301d-e954-4072-a88f-9759274142c8 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:09.566357 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67d5291-083d-45c4-814e-0f8f68fd2590 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:09.570055 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4f6cff-0656-41e3-aa57-ae073da205a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:09.570852 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f6cff-0656-41e3-aa57-ae073da205a9 None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:09.571209 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f6cff-0656-41e3-aa57-ae073da205a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:09.571601 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f6cff-0656-41e3-aa57-ae073da205a9 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:09.572567 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4f6cff-0656-41e3-aa57-ae073da205a9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:09.574179 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4f6cff-0656-41e3-aa57-ae073da205a9 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:09.603230 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8417aa-947e-4d1a-a9b5-60514465f2cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:09.603896 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8417aa-947e-4d1a-a9b5-60514465f2cf None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:09.604209 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8417aa-947e-4d1a-a9b5-60514465f2cf None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:09.604487 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8417aa-947e-4d1a-a9b5-60514465f2cf None admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:09.605848 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8417aa-947e-4d1a-a9b5-60514465f2cf None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:09.606145 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8417aa-947e-4d1a-a9b5-60514465f2cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:09.607552 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8417aa-947e-4d1a-a9b5-60514465f2cf None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:09.630782 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 441/867] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Jun 15 14:32:09 2022] POST /identity/v3/services => generated 251 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:32:09.630949 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 426/867] 10.222.0.42 () {56 vars in 1149 bytes} [Wed Jun 15 14:32:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:32:09.632355 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8417aa-947e-4d1a-a9b5-60514465f2cf None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:09.743728 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:09.752814 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8417aa-947e-4d1a-a9b5-60514465f2cf None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:09.800870 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccab67ed-1361-4034-ae3b-f478d5054739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:09.801819 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccab67ed-1361-4034-ae3b-f478d5054739 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:09.802311 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccab67ed-1361-4034-ae3b-f478d5054739 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:09.802823 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccab67ed-1361-4034-ae3b-f478d5054739 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:09.874945 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccab67ed-1361-4034-ae3b-f478d5054739 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:09.876782 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccab67ed-1361-4034-ae3b-f478d5054739 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:09.973750 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:09.987181 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 427/868] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jun 15 14:32:09 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 15 14:32:09.993198 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4f6cff-0656-41e3-aa57-ae073da205a9 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:10.104827 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4f6cff-0656-41e3-aa57-ae073da205a9 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:10.106369 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:10.140944 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f60289-4e36-4013-bb6d-c49a8b1b3c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:10.141742 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f60289-4e36-4013-bb6d-c49a8b1b3c92 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:10.142022 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f60289-4e36-4013-bb6d-c49a8b1b3c92 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:10.142298 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f60289-4e36-4013-bb6d-c49a8b1b3c92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:10.153845 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f60289-4e36-4013-bb6d-c49a8b1b3c92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:10.155522 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f60289-4e36-4013-bb6d-c49a8b1b3c92 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:10.173290 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-85f60289-4e36-4013-bb6d-c49a8b1b3c92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:10.173290 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:10.173290 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:10.173290 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:10.173290 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:10.173290 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:10.173290 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:10.173290 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:10.173290 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:10.173290 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:10.173290 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:10.173290 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:10.173290 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:32:10.173290 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:32:10.173290 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:32:10.181139 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:32:10.181139 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:10.181139 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:10.181139 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:10.181139 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:10.181139 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:10.181139 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:10.181139 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:10.181139 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:10.181139 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:10.181139 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:10.181139 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:10.181139 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:10.181139 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:10.181139 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:10.181139 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:10.181139 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:10.182030 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:32:10.182030 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:32:10.182030 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:10.182030 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:10.182030 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 442/869] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:32:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:10.182299 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccab67ed-1361-4034-ae3b-f478d5054739 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:10.288182 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccab67ed-1361-4034-ae3b-f478d5054739 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:10.324795 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467a4b2e-03bd-4846-9349-c9d9c30e9eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:10.325951 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467a4b2e-03bd-4846-9349-c9d9c30e9eda None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:10.326441 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467a4b2e-03bd-4846-9349-c9d9c30e9eda None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:10.326916 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467a4b2e-03bd-4846-9349-c9d9c30e9eda None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:10.327814 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467a4b2e-03bd-4846-9349-c9d9c30e9eda None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:10.328360 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467a4b2e-03bd-4846-9349-c9d9c30e9eda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:10.329902 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467a4b2e-03bd-4846-9349-c9d9c30e9eda None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:10.343240 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 428/870] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:32:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:10.348437 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f60289-4e36-4013-bb6d-c49a8b1b3c92 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:10.461354 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f60289-4e36-4013-bb6d-c49a8b1b3c92 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:10.486555 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49165617-2f11-4aa4-92c2-a4000c552f7a None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:10.486874 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49165617-2f11-4aa4-92c2-a4000c552f7a None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:10.487121 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49165617-2f11-4aa4-92c2-a4000c552f7a None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:10.488739 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 429/871] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:32:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:32:10.493655 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0799e799-101c-4f9e-85d8-c97084dd47bf None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:10.493943 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0799e799-101c-4f9e-85d8-c97084dd47bf None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:10.494259 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0799e799-101c-4f9e-85d8-c97084dd47bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:10.498109 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c933abcd-b90a-4784-8c64-65e2cc7e033b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:10.498916 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c933abcd-b90a-4784-8c64-65e2cc7e033b None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:10.499199 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c933abcd-b90a-4784-8c64-65e2cc7e033b None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:10.499632 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c933abcd-b90a-4784-8c64-65e2cc7e033b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:10.507327 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c933abcd-b90a-4784-8c64-65e2cc7e033b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:10.508801 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c933abcd-b90a-4784-8c64-65e2cc7e033b None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:10.514359 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-0799e799-101c-4f9e-85d8-c97084dd47bf None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:32:10.515636 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-0799e799-101c-4f9e-85d8-c97084dd47bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:32:10.524272 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-c933abcd-b90a-4784-8c64-65e2cc7e033b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:32:10.524272 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:10.524272 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:10.524272 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:10.524272 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:10.524272 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:10.524272 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:10.524272 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:10.524272 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:10.524272 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:10.524272 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:10.524272 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:10.524272 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:32:10.524272 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:32:10.524272 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:32:10.525153 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:32:10.525153 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:10.525153 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:10.525153 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:10.525153 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:10.525153 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:10.525153 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:10.525153 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:10.525153 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:10.525153 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:10.525153 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:10.525153 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:10.525153 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:10.525153 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:10.525153 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:10.525153 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:10.525153 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:10.526033 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:32:10.526033 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:32:10.526033 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:32:10.526033 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:32:10.526033 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:32:10.526033 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:32:10.526033 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:32:10.526033 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:10.526033 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 443/872] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jun 15 14:32:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:10.533078 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c933abcd-b90a-4784-8c64-65e2cc7e033b None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:10.564975 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:10.572913 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:10.600230 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 430/873] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:32:10 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:32:10.627149 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0799e799-101c-4f9e-85d8-c97084dd47bf None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:10.672760 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c933abcd-b90a-4784-8c64-65e2cc7e033b None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:10.711954 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c68bc22-5e66-487c-86a1-519bec08dfde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:10.712899 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c68bc22-5e66-487c-86a1-519bec08dfde None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:10.713332 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c68bc22-5e66-487c-86a1-519bec08dfde None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:10.713777 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c68bc22-5e66-487c-86a1-519bec08dfde None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:10.715582 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c68bc22-5e66-487c-86a1-519bec08dfde None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:10.716147 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c68bc22-5e66-487c-86a1-519bec08dfde None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:10.717859 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c68bc22-5e66-487c-86a1-519bec08dfde None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:10.731709 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 444/874] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jun 15 14:32:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:10.737804 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c68bc22-5e66-487c-86a1-519bec08dfde None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:10.749903 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0799e799-101c-4f9e-85d8-c97084dd47bf None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:10.793538 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d46eeb1-803f-4ef9-9df6-70d99ef52f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:10.794203 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d46eeb1-803f-4ef9-9df6-70d99ef52f3b None admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:10.794491 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d46eeb1-803f-4ef9-9df6-70d99ef52f3b None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:10.794763 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d46eeb1-803f-4ef9-9df6-70d99ef52f3b None admin] PATH_INFO: `/v3/services` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:10.795496 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d46eeb1-803f-4ef9-9df6-70d99ef52f3b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:10.796893 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d46eeb1-803f-4ef9-9df6-70d99ef52f3b None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:10.833743 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:10.864270 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c68bc22-5e66-487c-86a1-519bec08dfde None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:10.902848 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c694106c-eab3-4f2f-9552-74eb739e1fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:10.903502 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c694106c-eab3-4f2f-9552-74eb739e1fbe None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:10.903785 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c694106c-eab3-4f2f-9552-74eb739e1fbe None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:10.904063 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c694106c-eab3-4f2f-9552-74eb739e1fbe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:10.904734 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 431/875] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Jun 15 14:32:10 2022] POST /identity/v3/services => generated 269 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:32:10.904999 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c694106c-eab3-4f2f-9552-74eb739e1fbe None admin] RBAC: Adding query filter params (role.id=305a1ca2e6f740eba7dc696849e933eb, scope.project.id=3094ff8966f149bda784da6a51b1358f, user.id=19d82c04f9994654adfb10f4bd0573c2) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:10.905319 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c694106c-eab3-4f2f-9552-74eb739e1fbe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:10.906854 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c694106c-eab3-4f2f-9552-74eb739e1fbe None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:10.919347 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 445/876] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jun 15 14:32:10 2022] GET /identity/v3/role_assignments?user.id=19d82c04f9994654adfb10f4bd0573c2&scope.project.id=3094ff8966f149bda784da6a51b1358f&role.id=305a1ca2e6f740eba7dc696849e933eb => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:32:11.032987 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6a463-13a5-4d3e-8990-8a40384d5453 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:11.034056 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6a463-13a5-4d3e-8990-8a40384d5453 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:11.034567 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6a463-13a5-4d3e-8990-8a40384d5453 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:11.036711 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 432/877] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:32:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:32:11.042268 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9581baed-0d5f-4b96-82c4-ee25cf99f27a None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:11.043146 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9581baed-0d5f-4b96-82c4-ee25cf99f27a None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:11.043396 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9581baed-0d5f-4b96-82c4-ee25cf99f27a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:11.062558 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-9581baed-0d5f-4b96-82c4-ee25cf99f27a None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:32:11.063797 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-9581baed-0d5f-4b96-82c4-ee25cf99f27a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:32:11.150069 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 446/878] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:32:11 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:32:11.172656 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a6a463-13a5-4d3e-8990-8a40384d5453 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:11.223212 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:11.292297 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a6a463-13a5-4d3e-8990-8a40384d5453 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:11.330862 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546d6407-7cb2-4fe1-9bec-c35e795af7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:11.331675 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d6407-7cb2-4fe1-9bec-c35e795af7ac None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:11.331956 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d6407-7cb2-4fe1-9bec-c35e795af7ac None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:11.332263 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d6407-7cb2-4fe1-9bec-c35e795af7ac None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:11.332919 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546d6407-7cb2-4fe1-9bec-c35e795af7ac None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:11.334283 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546d6407-7cb2-4fe1-9bec-c35e795af7ac None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:11.347241 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-546d6407-7cb2-4fe1-9bec-c35e795af7ac None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 14:32:11.347241 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:11.347241 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:32:11.347241 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:32:11.347241 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:32:11.347241 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:32:11.347241 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:32:11.347241 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:32:11.347241 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 14:32:11.347241 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:32:11.347241 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:32:11.347241 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:32:11.347241 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:11.347241 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:11.347241 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:11.347241 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:11.347241 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:11.348323 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:11.348323 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:11.348323 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:11.348323 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:11.348323 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:11.348323 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:11.348323 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:32:11.348323 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:32:11.348323 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:32:11.348323 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:32:11.348323 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:11.348323 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:11.348323 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:11.348323 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:11.348323 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:11.348323 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:11.349171 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:11.349171 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:11.349171 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:11.349171 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:11.349171 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:11.349171 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:11.349171 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:11.349171 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:11.349171 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:11.349171 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:11.349171 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:32:11.349171 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:32:11.349171 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 14:32:11.349171 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:32:11.349171 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 433/879] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Jun 15 14:32:11 2022] GET /identity/v3/services/object-store => generated 93 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:11.357009 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9581baed-0d5f-4b96-82c4-ee25cf99f27a None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:11.423604 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:11.463395 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9581baed-0d5f-4b96-82c4-ee25cf99f27a None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:11.503138 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a43486-984f-4714-b93b-ecdf16d6336e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:11.504212 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a43486-984f-4714-b93b-ecdf16d6336e None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:11.504699 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a43486-984f-4714-b93b-ecdf16d6336e None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:11.505175 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a43486-984f-4714-b93b-ecdf16d6336e None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:11.506078 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a43486-984f-4714-b93b-ecdf16d6336e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:11.506581 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a43486-984f-4714-b93b-ecdf16d6336e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:11.508174 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a43486-984f-4714-b93b-ecdf16d6336e None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:11.514481 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 447/880] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Jun 15 14:32:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:11.520391 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546d6407-7cb2-4fe1-9bec-c35e795af7ac None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:11.637793 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546d6407-7cb2-4fe1-9bec-c35e795af7ac None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:11.640462 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:11.676273 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59420385-e7e6-4f07-97d3-d1d1f0dce0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:11.676867 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59420385-e7e6-4f07-97d3-d1d1f0dce0d1 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:11.677147 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59420385-e7e6-4f07-97d3-d1d1f0dce0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:11.677423 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59420385-e7e6-4f07-97d3-d1d1f0dce0d1 None admin] PATH_INFO: `/v3/services` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:11.678126 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59420385-e7e6-4f07-97d3-d1d1f0dce0d1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:11.678432 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59420385-e7e6-4f07-97d3-d1d1f0dce0d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:11.679783 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59420385-e7e6-4f07-97d3-d1d1f0dce0d1 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:11.685569 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 434/881] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Jun 15 14:32:11 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:12.005735 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b585c905-8413-427c-af6d-d4124ed32001 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:12.006079 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b585c905-8413-427c-af6d-d4124ed32001 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:12.006201 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b585c905-8413-427c-af6d-d4124ed32001 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:12.007928 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 448/882] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:32:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:32:12.013621 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3041794-5979-4ca7-bd66-1f4828ad8c8c None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:12.013972 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3041794-5979-4ca7-bd66-1f4828ad8c8c None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:12.014271 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3041794-5979-4ca7-bd66-1f4828ad8c8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:12.032830 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-f3041794-5979-4ca7-bd66-1f4828ad8c8c None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:32:12.034321 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-f3041794-5979-4ca7-bd66-1f4828ad8c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:32:12.110407 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 435/883] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:32:12 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:32:12.133543 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b585c905-8413-427c-af6d-d4124ed32001 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:12.254022 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b585c905-8413-427c-af6d-d4124ed32001 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:12.282617 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:12.290943 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b0ae56-6fe5-4ab8-8ede-179aa660ea79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:12.291746 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0ae56-6fe5-4ab8-8ede-179aa660ea79 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:12.292030 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0ae56-6fe5-4ab8-8ede-179aa660ea79 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:12.292333 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0ae56-6fe5-4ab8-8ede-179aa660ea79 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:12.307060 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b0ae56-6fe5-4ab8-8ede-179aa660ea79 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:12.307060 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b0ae56-6fe5-4ab8-8ede-179aa660ea79 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:12.307060 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-22b0ae56-6fe5-4ab8-8ede-179aa660ea79 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 14:32:12.307060 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:12.307060 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:32:12.307060 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:32:12.307060 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:32:12.307060 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:32:12.307060 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:32:12.307060 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:32:12.307060 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 14:32:12.307060 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:12.307060 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:32:12.307822 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:12.307822 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:12.307822 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:12.307822 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:12.307822 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:12.307822 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:12.307822 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:12.307822 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:12.307822 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:12.307822 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:12.307822 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:12.307822 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:12.307822 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:32:12.307822 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:32:12.307822 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:32:12.307822 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:32:12.307822 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:12.309188 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:12.309188 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:12.309188 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:12.309188 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:12.309188 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:12.309188 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:12.309188 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:12.309188 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:12.309188 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:12.309188 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:12.309188 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:12.309188 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:12.309188 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:12.309188 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:12.309188 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:12.309188 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:32:12.309188 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:32:12.311764 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 14:32:12.311764 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:12.311764 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 449/884] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Jun 15 14:32:12 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:12.318135 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3041794-5979-4ca7-bd66-1f4828ad8c8c None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:12.421531 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:12.426161 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3041794-5979-4ca7-bd66-1f4828ad8c8c None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:12.465690 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c362d793-e392-4b56-b7ef-81116124d35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:12.466353 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c362d793-e392-4b56-b7ef-81116124d35c None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:12.466634 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c362d793-e392-4b56-b7ef-81116124d35c None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:12.466909 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c362d793-e392-4b56-b7ef-81116124d35c None admin] PATH_INFO: `/v3/services` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:12.467607 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c362d793-e392-4b56-b7ef-81116124d35c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:12.467913 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c362d793-e392-4b56-b7ef-81116124d35c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:12.469296 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c362d793-e392-4b56-b7ef-81116124d35c None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:12.474527 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 436/885] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Jun 15 14:32:12 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:12.480813 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b0ae56-6fe5-4ab8-8ede-179aa660ea79 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:12.590963 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b0ae56-6fe5-4ab8-8ede-179aa660ea79 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:12.626346 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124bddd1-62ce-461d-9fef-c380c5dd5777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:12.627391 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124bddd1-62ce-461d-9fef-c380c5dd5777 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:12.627883 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124bddd1-62ce-461d-9fef-c380c5dd5777 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:12.628406 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124bddd1-62ce-461d-9fef-c380c5dd5777 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:12.629315 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124bddd1-62ce-461d-9fef-c380c5dd5777 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:12.629843 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124bddd1-62ce-461d-9fef-c380c5dd5777 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:12.631411 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124bddd1-62ce-461d-9fef-c380c5dd5777 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:12.637099 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 450/886] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Jun 15 14:32:12 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:12.645321 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c362d793-e392-4b56-b7ef-81116124d35c None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:12.764942 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c362d793-e392-4b56-b7ef-81116124d35c None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:12.800774 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4822ee4f-7925-4741-b0f7-4eb6e1aadc75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:12.801436 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4822ee4f-7925-4741-b0f7-4eb6e1aadc75 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:12.801784 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4822ee4f-7925-4741-b0f7-4eb6e1aadc75 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:12.802072 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4822ee4f-7925-4741-b0f7-4eb6e1aadc75 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:12.802877 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4822ee4f-7925-4741-b0f7-4eb6e1aadc75 None admin] RBAC: Adding query filter params (interface=public, service_id=711e712500544a2aa07c6b5dd12690d3, region_id=RegionOne) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:12.803197 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4822ee4f-7925-4741-b0f7-4eb6e1aadc75 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:12.804605 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4822ee4f-7925-4741-b0f7-4eb6e1aadc75 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:12.814114 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 437/887] 10.222.0.42 () {56 vars in 1251 bytes} [Wed Jun 15 14:32:12 2022] GET /identity/v3/endpoints?service_id=711e712500544a2aa07c6b5dd12690d3&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:12.819773 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124bddd1-62ce-461d-9fef-c380c5dd5777 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:12.881324 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:12.930974 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124bddd1-62ce-461d-9fef-c380c5dd5777 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:12.968868 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5f9d40-fd77-46c0-9743-7eb8cdc923bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:12.969543 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f9d40-fd77-46c0-9743-7eb8cdc923bd None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:12.969829 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f9d40-fd77-46c0-9743-7eb8cdc923bd None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:12.970110 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f9d40-fd77-46c0-9743-7eb8cdc923bd None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:12.970797 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5f9d40-fd77-46c0-9743-7eb8cdc923bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:12.972355 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5f9d40-fd77-46c0-9743-7eb8cdc923bd None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:12.978829 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 451/888] 10.222.0.42 () {56 vars in 1087 bytes} [Wed Jun 15 14:32:12 2022] GET /identity/v3/services => generated 1268 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:32:13.265978 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c050bc1-27a6-4ba2-b6e4-3471f2ffa4d4 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:13.267030 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c050bc1-27a6-4ba2-b6e4-3471f2ffa4d4 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:13.267536 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c050bc1-27a6-4ba2-b6e4-3471f2ffa4d4 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:13.269733 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 438/889] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:32:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:32:13.275787 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc80dc-9057-40f9-8a06-7f7dd83fef12 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:13.276676 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc80dc-9057-40f9-8a06-7f7dd83fef12 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:13.277256 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc80dc-9057-40f9-8a06-7f7dd83fef12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:13.295105 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-4fdc80dc-9057-40f9-8a06-7f7dd83fef12 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:32:13.296316 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-4fdc80dc-9057-40f9-8a06-7f7dd83fef12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:32:13.307070 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e420bff-0481-4d2f-a01c-cf9659f2136a None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:13.307396 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e420bff-0481-4d2f-a01c-cf9659f2136a None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:13.307623 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e420bff-0481-4d2f-a01c-cf9659f2136a None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:13.309886 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 439/890] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:32:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:32:13.315022 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f267a31a-e831-412c-a75c-bf96e6de02ea None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:13.315312 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f267a31a-e831-412c-a75c-bf96e6de02ea None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:13.315593 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f267a31a-e831-412c-a75c-bf96e6de02ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:13.332809 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-f267a31a-e831-412c-a75c-bf96e6de02ea None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:32:13.335116 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-f267a31a-e831-412c-a75c-bf96e6de02ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:32:13.360883 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:13.374652 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 452/891] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:32:13 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:32:13.396768 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdc80dc-9057-40f9-8a06-7f7dd83fef12 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:13.424285 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 440/892] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:32:13 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:32:13.442660 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f267a31a-e831-412c-a75c-bf96e6de02ea None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:13.455014 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:13.516073 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdc80dc-9057-40f9-8a06-7f7dd83fef12 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:13.565355 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f08f22-3bff-4aab-8595-c6d6941b1120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:13.566517 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f08f22-3bff-4aab-8595-c6d6941b1120 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:13.567024 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f08f22-3bff-4aab-8595-c6d6941b1120 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:13.567508 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f08f22-3bff-4aab-8595-c6d6941b1120 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:13.568500 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f08f22-3bff-4aab-8595-c6d6941b1120 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:13.570145 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f08f22-3bff-4aab-8595-c6d6941b1120 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:13.579814 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f267a31a-e831-412c-a75c-bf96e6de02ea None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:13.584515 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-e8f08f22-3bff-4aab-8595-c6d6941b1120 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 14:32:13.584515 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:13.584515 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:32:13.584515 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:32:13.584515 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:32:13.584515 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:32:13.584515 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:32:13.584515 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:32:13.584515 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 14:32:13.584515 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:13.584515 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:32:13.584515 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:13.584515 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:13.584515 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:13.584515 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:13.584515 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:13.585512 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:13.585512 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:13.585512 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:13.585512 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:13.585512 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:13.585512 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:13.585512 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:13.585512 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:32:13.585512 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:32:13.585512 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:32:13.585512 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:32:13.585512 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:13.585512 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:13.585512 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:13.585512 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:13.585512 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:13.586399 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:13.586399 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:13.586399 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:13.586399 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:13.586399 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:13.586399 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:13.586399 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:13.586399 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:13.586399 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:13.586399 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:13.586399 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:13.586399 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:32:13.586399 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:32:13.586399 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 14:32:13.586399 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:13.594611 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 453/893] 10.222.0.42 () {56 vars in 1132 bytes} [Wed Jun 15 14:32:13 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:32:13.596187 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f08f22-3bff-4aab-8595-c6d6941b1120 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:13.619370 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b3b7f8-3936-4bef-812a-e2629e39a99f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:13.620419 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b3b7f8-3936-4bef-812a-e2629e39a99f None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:13.622500 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b3b7f8-3936-4bef-812a-e2629e39a99f None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:13.622987 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b3b7f8-3936-4bef-812a-e2629e39a99f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:13.639867 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b3b7f8-3936-4bef-812a-e2629e39a99f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:32:13.651085 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-b2b3b7f8-3936-4bef-812a-e2629e39a99f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 14:32:13.651085 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:13.651085 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:13.651085 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:13.651085 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:13.651085 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:13.651085 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:13.651085 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:13.651085 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:13.651085 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:13.651085 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:13.651085 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:13.651085 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:32:13.651085 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:32:13.651085 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:32:13.651085 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:32:13.652453 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:32:13.652453 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:32:13.652453 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:32:13.652453 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:32:13.652453 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:13.652453 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:13.652453 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:13.652453 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:13.652453 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:13.652453 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:13.652453 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:13.652453 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:13.652453 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:13.652453 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:13.652453 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:13.652453 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:13.654437 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:13.654437 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:13.654437 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:13.654437 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:13.654437 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:32:13.654437 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:32:13.654437 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:32:13.654437 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:32:13.654437 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:32:13.654437 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:32:13.654437 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 14:32:13.654437 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:32:13.656166 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 441/894] 10.222.0.42 () {56 vars in 1120 bytes} [Wed Jun 15 14:32:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:13.658269 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b3b7f8-3936-4bef-812a-e2629e39a99f None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:13.678520 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:13.727213 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f08f22-3bff-4aab-8595-c6d6941b1120 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:13.746920 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:13.771482 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b3b7f8-3936-4bef-812a-e2629e39a99f None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:13.775280 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c6ce20-2ad1-4d32-a042-b10c924786ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:13.775969 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c6ce20-2ad1-4d32-a042-b10c924786ca None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:13.776374 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c6ce20-2ad1-4d32-a042-b10c924786ca None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:13.777357 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c6ce20-2ad1-4d32-a042-b10c924786ca None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:13.778089 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c6ce20-2ad1-4d32-a042-b10c924786ca None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:13.778403 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c6ce20-2ad1-4d32-a042-b10c924786ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:13.779888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c6ce20-2ad1-4d32-a042-b10c924786ca None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:13.788123 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 454/895] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Jun 15 14:32:13 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:32:13.795295 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c6ce20-2ad1-4d32-a042-b10c924786ca None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:13.817410 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd275ca1-7b1d-4098-b102-69b992de20ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:13.818468 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd275ca1-7b1d-4098-b102-69b992de20ef None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:13.818954 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd275ca1-7b1d-4098-b102-69b992de20ef None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:13.819431 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd275ca1-7b1d-4098-b102-69b992de20ef None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:13.820426 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd275ca1-7b1d-4098-b102-69b992de20ef None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:13.820930 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd275ca1-7b1d-4098-b102-69b992de20ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:13.822505 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd275ca1-7b1d-4098-b102-69b992de20ef None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:13.832124 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 442/896] 10.222.0.42 () {56 vars in 1115 bytes} [Wed Jun 15 14:32:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:13.837713 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd275ca1-7b1d-4098-b102-69b992de20ef None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:13.919487 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c6ce20-2ad1-4d32-a042-b10c924786ca None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:13.943587 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:13.957142 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b350ac8-bd92-492b-b4f9-e9ec4d40ba2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:13.957684 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b350ac8-bd92-492b-b4f9-e9ec4d40ba2c None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:13.957963 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b350ac8-bd92-492b-b4f9-e9ec4d40ba2c None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:13.958235 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b350ac8-bd92-492b-b4f9-e9ec4d40ba2c None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:13.958948 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b350ac8-bd92-492b-b4f9-e9ec4d40ba2c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:13.959248 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b350ac8-bd92-492b-b4f9-e9ec4d40ba2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:13.960638 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b350ac8-bd92-492b-b4f9-e9ec4d40ba2c None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:13.966130 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd275ca1-7b1d-4098-b102-69b992de20ef None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:13.968266 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 455/897] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Jun 15 14:32:13 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:13.968266 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65129b1e-52ed-4694-9ce5-0ca14beb0abc None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:13.968266 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65129b1e-52ed-4694-9ce5-0ca14beb0abc None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:13.968548 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65129b1e-52ed-4694-9ce5-0ca14beb0abc None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:13.970227 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 456/898] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:32:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:32:13.975303 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac47f7-6c7c-4985-b7ab-d7d82e6eb707 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:13.975609 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac47f7-6c7c-4985-b7ab-d7d82e6eb707 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:13.975888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac47f7-6c7c-4985-b7ab-d7d82e6eb707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:13.993383 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-45ac47f7-6c7c-4985-b7ab-d7d82e6eb707 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:32:13.994576 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-45ac47f7-6c7c-4985-b7ab-d7d82e6eb707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:32:14.005589 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0557a609-bc1b-4302-afed-fb6b21ff763f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:14.006682 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0557a609-bc1b-4302-afed-fb6b21ff763f None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:14.006972 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0557a609-bc1b-4302-afed-fb6b21ff763f None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:14.007250 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0557a609-bc1b-4302-afed-fb6b21ff763f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:14.019127 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0557a609-bc1b-4302-afed-fb6b21ff763f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:14.020619 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0557a609-bc1b-4302-afed-fb6b21ff763f None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:14.033572 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:14.038612 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-0557a609-bc1b-4302-afed-fb6b21ff763f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:14.038612 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:14.038612 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:14.038612 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:14.038612 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:14.038612 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:14.038612 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:14.038612 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:14.038612 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:14.038612 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:14.038612 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:14.038612 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:14.038612 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:32:14.038612 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:32:14.038612 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:32:14.039512 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:32:14.039512 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:14.039512 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:14.039512 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:14.039512 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:14.039512 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:14.039512 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:14.039512 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:14.039512 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:14.039512 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:14.039512 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:14.039512 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:14.039512 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:14.039512 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:14.039512 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:14.039512 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:14.039512 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:14.041250 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:32:14.041250 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:32:14.041250 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:14.041250 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:32:14.045023 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 443/899] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:32:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:14.049734 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0557a609-bc1b-4302-afed-fb6b21ff763f None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:14.076542 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 457/900] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:32:13 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:32:14.078418 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ac47f7-6c7c-4985-b7ab-d7d82e6eb707 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:14.165833 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0557a609-bc1b-4302-afed-fb6b21ff763f None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:14.188504 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ac47f7-6c7c-4985-b7ab-d7d82e6eb707 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:14.195979 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:14.202029 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4de22c-606f-4448-868b-5d2d0df3c0ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:14.202697 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4de22c-606f-4448-868b-5d2d0df3c0ba None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:14.202983 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4de22c-606f-4448-868b-5d2d0df3c0ba None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:14.203320 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4de22c-606f-4448-868b-5d2d0df3c0ba None admin] PATH_INFO: `/v3/endpoints` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:14.204217 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4de22c-606f-4448-868b-5d2d0df3c0ba None admin] RBAC: Adding query filter params (interface=public, service_id=14f0520639fc47e2b3e17ab5f5f151f6, region_id=RegionOne) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:14.204539 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4de22c-606f-4448-868b-5d2d0df3c0ba None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:14.205986 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4de22c-606f-4448-868b-5d2d0df3c0ba None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:14.216141 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 444/901] 10.222.0.42 () {56 vars in 1251 bytes} [Wed Jun 15 14:32:14 2022] GET /identity/v3/endpoints?service_id=14f0520639fc47e2b3e17ab5f5f151f6&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:14.216846 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4de22c-606f-4448-868b-5d2d0df3c0ba None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:14.226626 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f75686-d840-4848-b26b-1dd58f2a4b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:14.227618 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f75686-d840-4848-b26b-1dd58f2a4b83 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:14.227971 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f75686-d840-4848-b26b-1dd58f2a4b83 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:14.228284 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f75686-d840-4848-b26b-1dd58f2a4b83 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:14.229016 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f75686-d840-4848-b26b-1dd58f2a4b83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:14.229325 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f75686-d840-4848-b26b-1dd58f2a4b83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:14.231244 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f75686-d840-4848-b26b-1dd58f2a4b83 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:14.247925 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 458/902] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:32:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:14.247925 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f75686-d840-4848-b26b-1dd58f2a4b83 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:14.285403 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:14.326273 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4de22c-606f-4448-868b-5d2d0df3c0ba None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:14.358565 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f75686-d840-4848-b26b-1dd58f2a4b83 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:14.360880 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1e2e17-5694-4750-8035-ae1abe092d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:14.361544 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1e2e17-5694-4750-8035-ae1abe092d7d None admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:14.361830 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1e2e17-5694-4750-8035-ae1abe092d7d None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:14.362108 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1e2e17-5694-4750-8035-ae1abe092d7d None admin] PATH_INFO: `/v3/services` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:14.362826 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1e2e17-5694-4750-8035-ae1abe092d7d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:14.364228 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1e2e17-5694-4750-8035-ae1abe092d7d None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:14.393085 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2561ca-02f0-4c43-a307-20fa8b34efb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:14.393744 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2561ca-02f0-4c43-a307-20fa8b34efb3 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:14.394026 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2561ca-02f0-4c43-a307-20fa8b34efb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:14.394301 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2561ca-02f0-4c43-a307-20fa8b34efb3 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:14.394973 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2561ca-02f0-4c43-a307-20fa8b34efb3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:14.396467 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2561ca-02f0-4c43-a307-20fa8b34efb3 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:14.412225 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 459/903] 10.222.0.42 () {56 vars in 1087 bytes} [Wed Jun 15 14:32:14 2022] GET /identity/v3/services => generated 1268 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:32:14.412225 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2561ca-02f0-4c43-a307-20fa8b34efb3 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:14.445847 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 445/904] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Jun 15 14:32:14 2022] POST /identity/v3/services => generated 241 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:32:14.534055 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:14.545285 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2561ca-02f0-4c43-a307-20fa8b34efb3 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:14.588804 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71f7f81-e553-4cde-b26a-9bd5ca18529c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:14.590269 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71f7f81-e553-4cde-b26a-9bd5ca18529c None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:14.590529 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71f7f81-e553-4cde-b26a-9bd5ca18529c None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:14.590803 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71f7f81-e553-4cde-b26a-9bd5ca18529c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:14.602050 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71f7f81-e553-4cde-b26a-9bd5ca18529c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:14.603520 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71f7f81-e553-4cde-b26a-9bd5ca18529c None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:14.622703 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-f71f7f81-e553-4cde-b26a-9bd5ca18529c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 14:32:14.622703 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:14.622703 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:14.622703 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:14.622703 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:14.622703 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:14.622703 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:14.622703 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:14.622703 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:14.622703 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:14.622703 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:14.622703 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:14.622703 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:32:14.622703 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:32:14.622703 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:32:14.622703 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:32:14.623738 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:14.623738 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:14.623738 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:32:14.623738 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:32:14.623738 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:32:14.623738 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:32:14.623738 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:14.623738 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:14.623738 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:14.623738 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:14.623738 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:14.623738 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:14.623738 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:14.623738 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:14.623738 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:14.623738 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:14.623738 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:14.623738 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:14.625124 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:14.625124 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:14.625124 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:32:14.625124 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:32:14.625124 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:32:14.625124 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:32:14.625124 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:32:14.625124 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:32:14.625124 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 14:32:14.625124 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:14.625124 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 460/905] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Jun 15 14:32:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:14.629735 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1e2e17-5694-4750-8035-ae1abe092d7d None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:14.701019 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:14.748772 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1e2e17-5694-4750-8035-ae1abe092d7d None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:14.787234 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4113cb4b-9e4f-4dcb-9e4f-cf89fed8aade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:14.787979 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4113cb4b-9e4f-4dcb-9e4f-cf89fed8aade None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:14.788298 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4113cb4b-9e4f-4dcb-9e4f-cf89fed8aade None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:14.788584 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4113cb4b-9e4f-4dcb-9e4f-cf89fed8aade None admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:14.789478 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4113cb4b-9e4f-4dcb-9e4f-cf89fed8aade None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:14.789798 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4113cb4b-9e4f-4dcb-9e4f-cf89fed8aade None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:14.791233 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4113cb4b-9e4f-4dcb-9e4f-cf89fed8aade None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:14.809797 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 446/906] 10.222.0.42 () {56 vars in 1149 bytes} [Wed Jun 15 14:32:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:14.815111 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71f7f81-e553-4cde-b26a-9bd5ca18529c None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:14.922229 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71f7f81-e553-4cde-b26a-9bd5ca18529c None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:14.958450 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaadf98-595b-4cef-9e33-21d7e95c551b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:14.959096 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaadf98-595b-4cef-9e33-21d7e95c551b None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:14.959375 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaadf98-595b-4cef-9e33-21d7e95c551b None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:14.959647 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaadf98-595b-4cef-9e33-21d7e95c551b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:14.983717 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:15.032706 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaadf98-595b-4cef-9e33-21d7e95c551b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:15.034215 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaadf98-595b-4cef-9e33-21d7e95c551b None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:15.138822 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 461/907] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jun 15 14:32:14 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:32:15.145168 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4113cb4b-9e4f-4dcb-9e4f-cf89fed8aade None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:15.243001 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cfe1d6-fab3-4851-a269-41c7f3f9fc75 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:15.243721 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cfe1d6-fab3-4851-a269-41c7f3f9fc75 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:15.244245 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cfe1d6-fab3-4851-a269-41c7f3f9fc75 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:15.246608 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 462/908] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:32:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:32:15.252189 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0bade1-f581-4b58-9753-fc0e42a8f76d None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:15.252480 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0bade1-f581-4b58-9753-fc0e42a8f76d None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:15.253213 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0bade1-f581-4b58-9753-fc0e42a8f76d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:15.262710 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4113cb4b-9e4f-4dcb-9e4f-cf89fed8aade None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:15.275439 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-1a0bade1-f581-4b58-9753-fc0e42a8f76d None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:32:15.277857 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:15.278618 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-1a0bade1-f581-4b58-9753-fc0e42a8f76d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:32:15.301031 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b47398e-25e6-44cd-bc46-53b1c176e232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:15.301906 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b47398e-25e6-44cd-bc46-53b1c176e232 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:15.302187 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b47398e-25e6-44cd-bc46-53b1c176e232 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:15.302463 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b47398e-25e6-44cd-bc46-53b1c176e232 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:15.314862 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b47398e-25e6-44cd-bc46-53b1c176e232 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:15.316452 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b47398e-25e6-44cd-bc46-53b1c176e232 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:15.337076 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-3b47398e-25e6-44cd-bc46-53b1c176e232 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:15.337076 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:15.337076 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:15.337076 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:15.337076 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:15.337076 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:15.337076 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:15.337076 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:15.337076 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:15.337076 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:15.337076 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:15.337076 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:15.337076 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:32:15.337076 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:32:15.337076 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:32:15.338426 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:32:15.338426 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:15.338426 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:15.338426 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:15.338426 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:15.338426 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:15.338426 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:15.338426 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:15.338426 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:15.338426 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:15.338426 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:15.338426 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:15.338426 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:15.338426 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:15.338426 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:15.338426 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:15.338426 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:15.339357 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:32:15.339357 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:32:15.339357 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:15.339357 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:32:15.341186 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:15.346567 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 447/909] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:32:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:15.347830 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b47398e-25e6-44cd-bc46-53b1c176e232 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:15.368797 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 463/910] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:32:15 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:32:15.393304 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0bade1-f581-4b58-9753-fc0e42a8f76d None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:15.462414 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b47398e-25e6-44cd-bc46-53b1c176e232 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:15.502024 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ba1b6b-c770-4b01-8872-8daeaedf9f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:15.503235 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba1b6b-c770-4b01-8872-8daeaedf9f9c None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:15.503741 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba1b6b-c770-4b01-8872-8daeaedf9f9c None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:15.504258 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba1b6b-c770-4b01-8872-8daeaedf9f9c None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:15.505169 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ba1b6b-c770-4b01-8872-8daeaedf9f9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:15.505688 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ba1b6b-c770-4b01-8872-8daeaedf9f9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:15.505951 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0bade1-f581-4b58-9753-fc0e42a8f76d None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:15.507602 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ba1b6b-c770-4b01-8872-8daeaedf9f9c None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:15.521338 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 448/911] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:32:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:32:15.526841 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ba1b6b-c770-4b01-8872-8daeaedf9f9c None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:15.556462 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcafe9a9-1ea7-446a-aabf-36d1e0e24200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:15.557541 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcafe9a9-1ea7-446a-aabf-36d1e0e24200 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:15.558106 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcafe9a9-1ea7-446a-aabf-36d1e0e24200 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:15.559674 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcafe9a9-1ea7-446a-aabf-36d1e0e24200 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:15.560598 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcafe9a9-1ea7-446a-aabf-36d1e0e24200 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:15.562535 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcafe9a9-1ea7-446a-aabf-36d1e0e24200 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:15.565046 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:15.580129 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-fcafe9a9-1ea7-446a-aabf-36d1e0e24200 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 14:32:15.580129 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:15.580129 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:32:15.580129 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:32:15.580129 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:32:15.580129 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:32:15.580129 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:32:15.580129 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:32:15.580129 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 14:32:15.580129 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:15.580129 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:32:15.580129 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:15.580129 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:15.580129 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:15.580129 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:15.580129 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:15.580129 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:15.581131 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:15.581131 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:15.581131 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:15.581131 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:15.581131 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:15.581131 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:15.581131 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:32:15.581131 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:32:15.581131 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:32:15.581131 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:32:15.581131 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:15.581131 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:15.581131 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:15.581131 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:15.581131 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:15.581131 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:15.582043 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:15.582043 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:15.582043 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:15.582043 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:15.582043 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:15.582043 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:15.582043 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:15.582043 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:15.582043 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:15.582043 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:15.582043 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:32:15.582043 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:32:15.582043 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 14:32:15.582043 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:15.590634 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 464/912] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Jun 15 14:32:15 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:15.592461 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcafe9a9-1ea7-446a-aabf-36d1e0e24200 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:15.657095 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:15.660940 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ba1b6b-c770-4b01-8872-8daeaedf9f9c None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:15.704624 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e24f58c-3b58-4f26-8f5f-221e053c9c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:15.705654 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24f58c-3b58-4f26-8f5f-221e053c9c5a None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:15.705837 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24f58c-3b58-4f26-8f5f-221e053c9c5a None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:15.706068 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24f58c-3b58-4f26-8f5f-221e053c9c5a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:15.713893 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e24f58c-3b58-4f26-8f5f-221e053c9c5a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:15.715655 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e24f58c-3b58-4f26-8f5f-221e053c9c5a None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:15.727460 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcafe9a9-1ea7-446a-aabf-36d1e0e24200 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:15.731867 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-3e24f58c-3b58-4f26-8f5f-221e053c9c5a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:32:15.731867 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:15.731867 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:15.731867 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:15.731867 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:15.731867 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:15.731867 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:15.731867 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:15.731867 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:15.731867 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:15.731867 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:15.731867 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:15.731867 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:32:15.731867 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:32:15.731867 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:32:15.732966 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:32:15.732966 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:15.732966 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:15.732966 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:15.732966 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:15.732966 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:15.732966 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:15.732966 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:15.732966 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:15.732966 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:15.732966 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:15.732966 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:15.732966 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:15.732966 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:15.732966 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:15.732966 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:15.732966 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:15.733890 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:32:15.733890 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:32:15.733890 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:32:15.733890 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:32:15.733890 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:32:15.733890 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:32:15.733890 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:32:15.733890 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:32:15.737853 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 449/913] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jun 15 14:32:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:15.738692 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e24f58c-3b58-4f26-8f5f-221e053c9c5a None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:15.776908 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9c6be3-ae8a-4e1e-ba1d-9a84975d71d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:15.777961 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9c6be3-ae8a-4e1e-ba1d-9a84975d71d1 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:15.778503 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9c6be3-ae8a-4e1e-ba1d-9a84975d71d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:15.778977 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9c6be3-ae8a-4e1e-ba1d-9a84975d71d1 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:15.779899 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9c6be3-ae8a-4e1e-ba1d-9a84975d71d1 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:15.780442 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9c6be3-ae8a-4e1e-ba1d-9a84975d71d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:15.782585 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9c6be3-ae8a-4e1e-ba1d-9a84975d71d1 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:15.795138 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 465/914] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Jun 15 14:32:15 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:15.796770 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9c6be3-ae8a-4e1e-ba1d-9a84975d71d1 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:15.833101 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:15.848808 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e24f58c-3b58-4f26-8f5f-221e053c9c5a None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:15.885751 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb8ee0e-e708-4c08-9c36-19eab0113c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:15.886552 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb8ee0e-e708-4c08-9c36-19eab0113c48 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:15.886841 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb8ee0e-e708-4c08-9c36-19eab0113c48 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:15.887124 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb8ee0e-e708-4c08-9c36-19eab0113c48 None admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:15.888180 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb8ee0e-e708-4c08-9c36-19eab0113c48 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:15.888290 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb8ee0e-e708-4c08-9c36-19eab0113c48 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:15.889827 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb8ee0e-e708-4c08-9c36-19eab0113c48 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:15.902786 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 450/915] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jun 15 14:32:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:15.909108 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb8ee0e-e708-4c08-9c36-19eab0113c48 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:15.914266 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9c6be3-ae8a-4e1e-ba1d-9a84975d71d1 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:15.926278 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:15.951566 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980eb9da-de1b-426a-8d6d-a93e94ccdeb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:15.952509 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980eb9da-de1b-426a-8d6d-a93e94ccdeb0 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:15.952973 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980eb9da-de1b-426a-8d6d-a93e94ccdeb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:15.953603 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980eb9da-de1b-426a-8d6d-a93e94ccdeb0 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:15.954457 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980eb9da-de1b-426a-8d6d-a93e94ccdeb0 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:15.954907 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980eb9da-de1b-426a-8d6d-a93e94ccdeb0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:15.956449 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980eb9da-de1b-426a-8d6d-a93e94ccdeb0 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:15.967278 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 466/916] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Jun 15 14:32:15 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:15.971162 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980eb9da-de1b-426a-8d6d-a93e94ccdeb0 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:16.026521 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb8ee0e-e708-4c08-9c36-19eab0113c48 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:16.063753 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89fada3-ff5a-4dd1-be03-817402878021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:16.064712 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89fada3-ff5a-4dd1-be03-817402878021 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:16.065172 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89fada3-ff5a-4dd1-be03-817402878021 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:16.065594 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89fada3-ff5a-4dd1-be03-817402878021 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:16.066572 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89fada3-ff5a-4dd1-be03-817402878021 None admin] RBAC: Adding query filter params (role.id=76f33d89f76347be9ef2fdac0435173f, scope.project.id=3094ff8966f149bda784da6a51b1358f, user.id=19d82c04f9994654adfb10f4bd0573c2) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:16.067032 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89fada3-ff5a-4dd1-be03-817402878021 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:16.068633 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89fada3-ff5a-4dd1-be03-817402878021 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:16.075691 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 451/917] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jun 15 14:32:15 2022] GET /identity/v3/role_assignments?user.id=19d82c04f9994654adfb10f4bd0573c2&scope.project.id=3094ff8966f149bda784da6a51b1358f&role.id=76f33d89f76347be9ef2fdac0435173f => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:16.106628 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980eb9da-de1b-426a-8d6d-a93e94ccdeb0 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:16.148000 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e58d62e-4083-4d78-b771-0cc2312296a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:16.149219 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e58d62e-4083-4d78-b771-0cc2312296a1 None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:16.149676 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e58d62e-4083-4d78-b771-0cc2312296a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:16.150119 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e58d62e-4083-4d78-b771-0cc2312296a1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:16.151042 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e58d62e-4083-4d78-b771-0cc2312296a1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:16.153310 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e58d62e-4083-4d78-b771-0cc2312296a1 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:16.224560 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 467/918] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Jun 15 14:32:15 2022] POST /identity/v3/endpoints => generated 346 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:32:16.695952 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c507b25-7269-4523-8593-4e1a6d2a3476 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:16.696520 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c507b25-7269-4523-8593-4e1a6d2a3476 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:16.696520 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c507b25-7269-4523-8593-4e1a6d2a3476 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:16.698034 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 452/919] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:32:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:32:16.704337 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b149a-9ead-4280-8a10-06f25952451f None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:16.704761 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b149a-9ead-4280-8a10-06f25952451f None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:16.704966 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b149a-9ead-4280-8a10-06f25952451f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:16.723914 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-659b149a-9ead-4280-8a10-06f25952451f None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:32:16.725181 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-659b149a-9ead-4280-8a10-06f25952451f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:32:16.753963 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:16.785539 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430930aa-e905-4392-a608-cf93254f7f1b None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:16.785898 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430930aa-e905-4392-a608-cf93254f7f1b None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:16.786106 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430930aa-e905-4392-a608-cf93254f7f1b None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:16.787704 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 453/920] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:32:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:32:16.793594 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90774f58-78ee-443f-9a38-bfdb8a62690c None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:16.793950 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90774f58-78ee-443f-9a38-bfdb8a62690c None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:16.794309 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90774f58-78ee-443f-9a38-bfdb8a62690c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:16.804960 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 468/921] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:32:16 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:32:16.812321 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-90774f58-78ee-443f-9a38-bfdb8a62690c None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:32:16.813513 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-90774f58-78ee-443f-9a38-bfdb8a62690c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:32:16.815641 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:16.829273 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659b149a-9ead-4280-8a10-06f25952451f None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:16.895174 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 454/922] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:32:16 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:32:16.919878 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90774f58-78ee-443f-9a38-bfdb8a62690c None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:16.940637 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659b149a-9ead-4280-8a10-06f25952451f None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:16.979010 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d0208d-f2b9-4d0b-bf9a-79fb38db83a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:16.980570 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d0208d-f2b9-4d0b-bf9a-79fb38db83a1 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:16.980866 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d0208d-f2b9-4d0b-bf9a-79fb38db83a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:16.981193 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d0208d-f2b9-4d0b-bf9a-79fb38db83a1 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:16.981856 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d0208d-f2b9-4d0b-bf9a-79fb38db83a1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:16.983228 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d0208d-f2b9-4d0b-bf9a-79fb38db83a1 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:16.997268 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-a3d0208d-f2b9-4d0b-bf9a-79fb38db83a1 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 14:32:16.997268 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:16.997268 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:32:16.997268 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:32:16.997268 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:32:16.997268 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:32:16.997268 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:32:16.997268 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:32:16.997268 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 14:32:16.997268 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:16.997268 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:32:16.997268 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:16.997268 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:16.997268 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:16.997268 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:16.997268 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:16.998347 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:16.998347 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:16.998347 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:16.998347 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:16.998347 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:16.998347 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:16.998347 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:16.998347 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:32:16.998347 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:32:16.998347 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:32:16.998347 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:32:16.998347 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:16.998347 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:16.998347 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:16.998347 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:16.998347 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:16.999210 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:16.999210 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:16.999210 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:16.999210 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:16.999210 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:16.999210 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:16.999210 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:16.999210 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:16.999210 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:16.999210 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:16.999210 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:16.999210 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:32:16.999210 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:32:16.999210 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 14:32:16.999210 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:16.999210 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 469/923] 10.222.0.42 () {56 vars in 1132 bytes} [Wed Jun 15 14:32:16 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:32:17.005895 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d0208d-f2b9-4d0b-bf9a-79fb38db83a1 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:17.025524 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90774f58-78ee-443f-9a38-bfdb8a62690c None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:17.030114 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:17.062098 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d8be11-fd0c-4b6c-89f1-d479a87491cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:17.062922 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d8be11-fd0c-4b6c-89f1-d479a87491cc None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:17.063271 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d8be11-fd0c-4b6c-89f1-d479a87491cc None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:17.063639 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d8be11-fd0c-4b6c-89f1-d479a87491cc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:17.064393 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d8be11-fd0c-4b6c-89f1-d479a87491cc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:17.065838 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d8be11-fd0c-4b6c-89f1-d479a87491cc None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:17.078479 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-42d8be11-fd0c-4b6c-89f1-d479a87491cc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 14:32:17.078479 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:17.078479 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:32:17.078479 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:32:17.078479 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:32:17.078479 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:32:17.078479 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:32:17.078479 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:32:17.078479 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 14:32:17.078479 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:32:17.078479 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:32:17.078479 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:32:17.078479 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:17.078479 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:17.078479 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:17.078479 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:17.078479 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:17.080633 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:17.080633 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:17.080633 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:17.080633 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:17.080633 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:17.080633 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:17.080633 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:32:17.080633 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:32:17.080633 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:32:17.080633 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:32:17.080633 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:17.080633 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:17.080633 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:17.080633 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:17.080633 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:17.080633 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:17.082613 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:17.082613 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:17.082613 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:17.082613 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:17.082613 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:17.082613 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:17.082613 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:17.082613 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:17.082613 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:17.082613 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:17.082613 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:32:17.082613 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:32:17.082613 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 14:32:17.082613 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:32:17.088126 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 455/924] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Jun 15 14:32:16 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:32:17.088126 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d8be11-fd0c-4b6c-89f1-d479a87491cc None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:17.121375 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d0208d-f2b9-4d0b-bf9a-79fb38db83a1 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:17.134352 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:17.157641 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7f682c-8224-4b85-9dd7-010c88598118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:17.158306 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7f682c-8224-4b85-9dd7-010c88598118 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:17.161289 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7f682c-8224-4b85-9dd7-010c88598118 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:17.161289 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7f682c-8224-4b85-9dd7-010c88598118 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:17.161289 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7f682c-8224-4b85-9dd7-010c88598118 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:17.161289 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7f682c-8224-4b85-9dd7-010c88598118 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:17.161880 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7f682c-8224-4b85-9dd7-010c88598118 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:17.172173 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 470/925] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Jun 15 14:32:17 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:32:17.173142 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7f682c-8224-4b85-9dd7-010c88598118 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:17.189133 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d8be11-fd0c-4b6c-89f1-d479a87491cc None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:17.224453 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1772640-eefc-4ad9-ae1f-9cc0cf8cec4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:17.225117 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1772640-eefc-4ad9-ae1f-9cc0cf8cec4d None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:17.225398 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1772640-eefc-4ad9-ae1f-9cc0cf8cec4d None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:17.225676 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1772640-eefc-4ad9-ae1f-9cc0cf8cec4d None admin] PATH_INFO: `/v3/services` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:17.226388 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1772640-eefc-4ad9-ae1f-9cc0cf8cec4d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:17.226696 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1772640-eefc-4ad9-ae1f-9cc0cf8cec4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:17.228112 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1772640-eefc-4ad9-ae1f-9cc0cf8cec4d None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:17.239015 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 456/926] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Jun 15 14:32:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:17.240531 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1772640-eefc-4ad9-ae1f-9cc0cf8cec4d None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:17.266896 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:17.284482 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7f682c-8224-4b85-9dd7-010c88598118 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:17.309672 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:17.323502 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36e3c04-3f37-4053-8755-55406c057522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:17.324717 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e3c04-3f37-4053-8755-55406c057522 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:17.325156 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e3c04-3f37-4053-8755-55406c057522 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:17.325607 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e3c04-3f37-4053-8755-55406c057522 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:17.326528 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36e3c04-3f37-4053-8755-55406c057522 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:17.326996 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36e3c04-3f37-4053-8755-55406c057522 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:17.328804 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36e3c04-3f37-4053-8755-55406c057522 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:17.339665 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 471/927] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Jun 15 14:32:17 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:17.340756 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36e3c04-3f37-4053-8755-55406c057522 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:17.345426 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1772640-eefc-4ad9-ae1f-9cc0cf8cec4d None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:17.380386 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6d186a-9f47-42ab-9bb3-7d2861e33d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:17.381048 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6d186a-9f47-42ab-9bb3-7d2861e33d46 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:17.381334 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6d186a-9f47-42ab-9bb3-7d2861e33d46 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:17.381610 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6d186a-9f47-42ab-9bb3-7d2861e33d46 None admin] PATH_INFO: `/v3/services` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:17.382320 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6d186a-9f47-42ab-9bb3-7d2861e33d46 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:17.382627 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6d186a-9f47-42ab-9bb3-7d2861e33d46 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:17.383991 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6d186a-9f47-42ab-9bb3-7d2861e33d46 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:17.394138 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 457/928] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Jun 15 14:32:17 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:32:17.395334 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6d186a-9f47-42ab-9bb3-7d2861e33d46 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:17.453228 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36e3c04-3f37-4053-8755-55406c057522 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:17.491626 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8403b0d8-37a0-4804-800f-3e537eca2056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:17.492640 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8403b0d8-37a0-4804-800f-3e537eca2056 None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:17.493076 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8403b0d8-37a0-4804-800f-3e537eca2056 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:17.493529 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8403b0d8-37a0-4804-800f-3e537eca2056 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:17.494418 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8403b0d8-37a0-4804-800f-3e537eca2056 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:17.496595 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8403b0d8-37a0-4804-800f-3e537eca2056 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:17.502856 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6d186a-9f47-42ab-9bb3-7d2861e33d46 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:17.533024 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:17.544231 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e426efc-2b6c-4273-9e3c-33f88bd1f6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:17.544992 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e426efc-2b6c-4273-9e3c-33f88bd1f6d3 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:17.545279 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e426efc-2b6c-4273-9e3c-33f88bd1f6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:17.545558 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e426efc-2b6c-4273-9e3c-33f88bd1f6d3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:17.546358 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e426efc-2b6c-4273-9e3c-33f88bd1f6d3 None admin] RBAC: Adding query filter params (interface=public, service_id=b80b04e510564033a7f22aa02a862777, region_id=RegionOne) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:17.546675 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e426efc-2b6c-4273-9e3c-33f88bd1f6d3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:17.548032 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e426efc-2b6c-4273-9e3c-33f88bd1f6d3 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:17.556741 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 458/929] 10.222.0.42 () {56 vars in 1251 bytes} [Wed Jun 15 14:32:17 2022] GET /identity/v3/endpoints?service_id=b80b04e510564033a7f22aa02a862777&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:32:17.563003 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e426efc-2b6c-4273-9e3c-33f88bd1f6d3 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:17.621727 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 472/930] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Jun 15 14:32:17 2022] POST /identity/v3/endpoints => generated 347 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:32:17.681208 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e426efc-2b6c-4273-9e3c-33f88bd1f6d3 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:17.726855 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83d6589-3d2d-4ba2-83ee-03273e4bfdb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:17.727524 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83d6589-3d2d-4ba2-83ee-03273e4bfdb0 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:17.727798 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83d6589-3d2d-4ba2-83ee-03273e4bfdb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:17.728078 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83d6589-3d2d-4ba2-83ee-03273e4bfdb0 None admin] PATH_INFO: `/v3/services` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:17.728792 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83d6589-3d2d-4ba2-83ee-03273e4bfdb0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:17.730949 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83d6589-3d2d-4ba2-83ee-03273e4bfdb0 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:17.742816 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 459/931] 10.222.0.42 () {56 vars in 1087 bytes} [Wed Jun 15 14:32:17 2022] GET /identity/v3/services => generated 1497 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:32:18.304831 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c8b432-7318-4002-bc43-28d23aa035d6 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:18.305383 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c8b432-7318-4002-bc43-28d23aa035d6 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:18.305383 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c8b432-7318-4002-bc43-28d23aa035d6 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:18.306961 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 473/932] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:32:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:32:18.312485 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8a5e84-e019-4505-ad40-506e5d41c8ee None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:18.312770 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8a5e84-e019-4505-ad40-506e5d41c8ee None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:18.313047 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8a5e84-e019-4505-ad40-506e5d41c8ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:18.330424 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-ee8a5e84-e019-4505-ad40-506e5d41c8ee None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:32:18.331591 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-ee8a5e84-e019-4505-ad40-506e5d41c8ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:32:18.382425 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:18.400129 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 460/933] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:32:18 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:32:18.425060 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c8b432-7318-4002-bc43-28d23aa035d6 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:18.429399 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:18.484873 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b8a39f-70ac-4ae0-bcca-ffda2639b12d None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:18.485299 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b8a39f-70ac-4ae0-bcca-ffda2639b12d None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:18.485693 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b8a39f-70ac-4ae0-bcca-ffda2639b12d None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:18.487166 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 461/934] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:32:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:32:18.492525 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f17fb9-4eaa-41a3-86e6-caa5a1ec0196 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:18.492910 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f17fb9-4eaa-41a3-86e6-caa5a1ec0196 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:18.493717 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f17fb9-4eaa-41a3-86e6-caa5a1ec0196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:18.510845 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-b4f17fb9-4eaa-41a3-86e6-caa5a1ec0196 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:32:18.513013 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-b4f17fb9-4eaa-41a3-86e6-caa5a1ec0196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:32:18.544570 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c8b432-7318-4002-bc43-28d23aa035d6 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:18.581929 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33328984-73d8-4efc-88bd-9c4646875bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:18.583026 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33328984-73d8-4efc-88bd-9c4646875bf7 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:18.583503 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33328984-73d8-4efc-88bd-9c4646875bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:18.583954 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33328984-73d8-4efc-88bd-9c4646875bf7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:18.589435 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 462/935] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:32:18 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:32:18.600271 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33328984-73d8-4efc-88bd-9c4646875bf7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:32:18.609163 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-33328984-73d8-4efc-88bd-9c4646875bf7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 14:32:18.609163 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:18.609163 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:18.609163 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:18.609163 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:18.609163 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:18.609163 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:18.609163 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:18.609163 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:18.609163 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:18.609163 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:18.609163 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:18.609163 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:32:18.609163 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:32:18.609163 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:32:18.609163 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:32:18.610103 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:32:18.610103 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:32:18.610103 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:32:18.610103 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:32:18.610103 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:18.610103 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:18.610103 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:18.610103 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:18.610103 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:18.610103 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:18.610103 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:18.610103 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:18.610103 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:18.610103 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:18.610103 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:18.610103 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:18.610953 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:18.610953 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:18.610953 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:18.610953 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:18.610953 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:32:18.610953 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:32:18.610953 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:32:18.610953 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:32:18.610953 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:32:18.610953 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:32:18.610953 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 14:32:18.610953 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:18.610953 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 474/936] 10.222.0.42 () {56 vars in 1120 bytes} [Wed Jun 15 14:32:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:18.616499 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f17fb9-4eaa-41a3-86e6-caa5a1ec0196 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:18.621597 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33328984-73d8-4efc-88bd-9c4646875bf7 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:18.688937 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:18.729260 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33328984-73d8-4efc-88bd-9c4646875bf7 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:18.740877 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f17fb9-4eaa-41a3-86e6-caa5a1ec0196 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:18.757063 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:18.771870 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107100df-d29e-4fb0-843c-329694612772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:18.772697 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107100df-d29e-4fb0-843c-329694612772 None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:18.772983 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107100df-d29e-4fb0-843c-329694612772 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:18.773425 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107100df-d29e-4fb0-843c-329694612772 None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:18.774478 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107100df-d29e-4fb0-843c-329694612772 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:18.776684 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107100df-d29e-4fb0-843c-329694612772 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:18.778152 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: WARNING keystone.server.flask.common [None req-107100df-d29e-4fb0-843c-329694612772 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 15 14:32:18.780041 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c8ce7e-4a1b-4120-b680-1042360583e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:18.780917 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c8ce7e-4a1b-4120-b680-1042360583e6 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:18.781261 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c8ce7e-4a1b-4120-b680-1042360583e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:18.781677 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c8ce7e-4a1b-4120-b680-1042360583e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:18.782584 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c8ce7e-4a1b-4120-b680-1042360583e6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:18.782956 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c8ce7e-4a1b-4120-b680-1042360583e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:18.784392 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c8ce7e-4a1b-4120-b680-1042360583e6 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:18.795222 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 463/937] 10.222.0.42 () {56 vars in 1115 bytes} [Wed Jun 15 14:32:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:18.799577 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c8ce7e-4a1b-4120-b680-1042360583e6 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:18.838603 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.sql.core [None req-107100df-d29e-4fb0-843c-329694612772 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 15 14:32:18.838603 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 15 14:32:18.838603 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [parameters: {'id': 'f7bcc65ebd094a8ca23f7092b30981cf', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 15 14:32:18.838603 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80600) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 15 14:32:18.849982 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-107100df-d29e-4fb0-843c-329694612772 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f7bcc65ebd094a8ca23f7092b30981cf.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f7bcc65ebd094a8ca23f7092b30981cf. Jun 15 14:32:18.849982 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:18.849982 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 15 14:32:18.849982 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 15 14:32:18.849982 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 15 14:32:18.849982 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 15 14:32:18.849982 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 15 14:32:18.849982 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application result = self._query(query) Jun 15 14:32:18.849982 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 15 14:32:18.849982 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application conn.query(q) Jun 15 14:32:18.849982 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 15 14:32:18.849982 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 15 14:32:18.849982 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 15 14:32:18.850857 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application result.read() Jun 15 14:32:18.850857 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 15 14:32:18.850857 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 15 14:32:18.850857 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 15 14:32:18.850857 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 15 14:32:18.850857 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 15 14:32:18.850857 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 15 14:32:18.850857 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 15 14:32:18.850857 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 15 14:32:18.850857 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 15 14:32:18.850857 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:18.850857 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 15 14:32:18.850857 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:18.850857 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:18.850857 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 15 14:32:18.850857 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 15 14:32:18.850857 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 15 14:32:18.850857 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 15 14:32:18.851851 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 15 14:32:18.851851 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application next(self.gen) Jun 15 14:32:18.851851 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 15 14:32:18.851851 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application yield resource Jun 15 14:32:18.851851 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 15 14:32:18.851851 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application next(self.gen) Jun 15 14:32:18.851851 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 15 14:32:18.851851 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self.session.rollback() Jun 15 14:32:18.851851 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 15 14:32:18.851851 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self.force_reraise() Jun 15 14:32:18.851851 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 15 14:32:18.851851 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise self.value Jun 15 14:32:18.851851 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 15 14:32:18.851851 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 15 14:32:18.851851 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 15 14:32:18.851851 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application session.commit() Jun 15 14:32:18.851851 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 15 14:32:18.852780 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 15 14:32:18.852780 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 15 14:32:18.852780 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._prepare_impl() Jun 15 14:32:18.852780 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 15 14:32:18.852780 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self.session.flush() Jun 15 14:32:18.852780 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 15 14:32:18.852780 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._flush(objects) Jun 15 14:32:18.852780 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 15 14:32:18.852780 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 15 14:32:18.852780 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 15 14:32:18.852780 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application compat.raise_( Jun 15 14:32:18.852780 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 15 14:32:18.852780 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception Jun 15 14:32:18.852780 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 15 14:32:18.852780 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application flush_context.execute() Jun 15 14:32:18.852780 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 15 14:32:18.852780 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rec.execute(self) Jun 15 14:32:18.853734 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 15 14:32:18.853734 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 15 14:32:18.853734 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 15 14:32:18.853734 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 15 14:32:18.853734 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 15 14:32:18.853734 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 15 14:32:18.853734 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 15 14:32:18.853734 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 15 14:32:18.853734 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 15 14:32:18.853734 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 15 14:32:18.853734 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 15 14:32:18.853734 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 15 14:32:18.853734 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 15 14:32:18.853734 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 15 14:32:18.853734 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 15 14:32:18.854531 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 15 14:32:18.854531 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 15 14:32:18.854531 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception Jun 15 14:32:18.854531 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 15 14:32:18.854531 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 15 14:32:18.854531 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 15 14:32:18.854531 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 15 14:32:18.854531 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 15 14:32:18.854531 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application result = self._query(query) Jun 15 14:32:18.854531 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 15 14:32:18.854531 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application conn.query(q) Jun 15 14:32:18.854531 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 15 14:32:18.854531 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 15 14:32:18.854531 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 15 14:32:18.854531 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application result.read() Jun 15 14:32:18.854531 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 15 14:32:18.854531 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 15 14:32:18.855938 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 15 14:32:18.855938 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 15 14:32:18.855938 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 15 14:32:18.855938 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 15 14:32:18.855938 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 15 14:32:18.855938 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 15 14:32:18.855938 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 15 14:32:18.855938 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 15 14:32:18.855938 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [parameters: {'id': 'f7bcc65ebd094a8ca23f7092b30981cf', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 15 14:32:18.855938 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 15 14:32:18.855938 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:18.855938 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:32:18.855938 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:18.855938 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:18.855938 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 15 14:32:18.857017 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 15 14:32:18.857017 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 15 14:32:18.857017 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 15 14:32:18.857017 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 15 14:32:18.857017 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:18.857017 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:32:18.857017 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:18.857017 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:18.857017 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:18.857017 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:18.857017 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:18.857017 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:18.857017 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:18.857017 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:18.857017 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:18.857017 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:18.857017 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:18.859174 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:18.859174 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 15 14:32:18.859174 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 15 14:32:18.859174 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:18.859174 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:18.859174 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 15 14:32:18.859174 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 15 14:32:18.859174 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f7bcc65ebd094a8ca23f7092b30981cf. Jun 15 14:32:18.859174 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:18.862197 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 475/938] 10.222.0.42 () {60 vars in 1140 bytes} [Wed Jun 15 14:32:18 2022] POST /identity/v3/projects => generated 262 bytes in 241 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:18.868981 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107100df-d29e-4fb0-843c-329694612772 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:18.931580 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c8ce7e-4a1b-4120-b680-1042360583e6 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:18.979769 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc7cb72-b9cc-4bdb-9c8f-5e10c00b694a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:18.981595 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc7cb72-b9cc-4bdb-9c8f-5e10c00b694a None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:18.981884 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc7cb72-b9cc-4bdb-9c8f-5e10c00b694a None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:18.982425 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc7cb72-b9cc-4bdb-9c8f-5e10c00b694a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:18.989222 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107100df-d29e-4fb0-843c-329694612772 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:18.998908 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc7cb72-b9cc-4bdb-9c8f-5e10c00b694a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:19.001179 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc7cb72-b9cc-4bdb-9c8f-5e10c00b694a None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:19.013131 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:19.027765 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-6cc7cb72-b9cc-4bdb-9c8f-5e10c00b694a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:19.027765 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:19.027765 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:19.027765 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:19.027765 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:19.027765 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:19.027765 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:19.027765 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:19.027765 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:19.027765 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:19.027765 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:19.027765 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:19.027765 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:32:19.027765 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:32:19.027765 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:32:19.028918 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:32:19.028918 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:19.028918 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:19.028918 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:19.028918 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:19.028918 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:19.028918 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:19.028918 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:19.028918 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:19.028918 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:19.028918 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:19.028918 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:19.028918 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:19.028918 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:19.028918 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:19.028918 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:19.028918 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:19.030033 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:32:19.030033 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:32:19.030033 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:19.030033 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:32:19.030517 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420a5580-104d-49f6-9a49-6ea2e1d8f5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:19.031413 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a5580-104d-49f6-9a49-6ea2e1d8f5f2 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:19.031972 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a5580-104d-49f6-9a49-6ea2e1d8f5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:19.032328 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a5580-104d-49f6-9a49-6ea2e1d8f5f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:19.032481 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 464/939] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:32:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:32:19.044140 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc7cb72-b9cc-4bdb-9c8f-5e10c00b694a None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:19.044303 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420a5580-104d-49f6-9a49-6ea2e1d8f5f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:19.044303 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420a5580-104d-49f6-9a49-6ea2e1d8f5f2 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:19.059504 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-420a5580-104d-49f6-9a49-6ea2e1d8f5f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:32:19.059504 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:19.059504 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:19.059504 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:19.059504 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:19.059504 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:19.059504 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:19.059504 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:19.059504 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:19.059504 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:19.059504 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:19.059504 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:19.059504 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:32:19.059504 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:32:19.059504 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:32:19.060456 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:32:19.060456 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:19.060456 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:19.060456 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:19.060456 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:19.060456 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:19.060456 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:19.060456 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:19.060456 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:19.060456 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:19.060456 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:19.060456 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:19.060456 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:19.060456 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:19.060456 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:19.060456 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:19.060456 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:19.061337 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:32:19.061337 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:32:19.061337 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:32:19.061337 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:32:19.061337 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:32:19.061337 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:32:19.061337 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:32:19.061337 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:19.063802 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 476/940] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jun 15 14:32:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:19.069872 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420a5580-104d-49f6-9a49-6ea2e1d8f5f2 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:19.080143 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:19.159639 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc7cb72-b9cc-4bdb-9c8f-5e10c00b694a None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:19.180009 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420a5580-104d-49f6-9a49-6ea2e1d8f5f2 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:19.194457 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd9494b-a1cb-446b-823e-7b972d32bfbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:19.195062 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd9494b-a1cb-446b-823e-7b972d32bfbe None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:19.195349 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd9494b-a1cb-446b-823e-7b972d32bfbe None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:19.195627 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd9494b-a1cb-446b-823e-7b972d32bfbe None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:19.196438 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd9494b-a1cb-446b-823e-7b972d32bfbe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:19.196685 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd9494b-a1cb-446b-823e-7b972d32bfbe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:19.198046 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd9494b-a1cb-446b-823e-7b972d32bfbe None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:19.211142 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 465/941] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:32:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:19.217200 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd9494b-a1cb-446b-823e-7b972d32bfbe None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:19.223683 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea389467-1ae3-4924-9d4d-6be4b95bbe3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:19.224370 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea389467-1ae3-4924-9d4d-6be4b95bbe3b None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:19.224652 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea389467-1ae3-4924-9d4d-6be4b95bbe3b None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:19.224930 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea389467-1ae3-4924-9d4d-6be4b95bbe3b None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:19.225738 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea389467-1ae3-4924-9d4d-6be4b95bbe3b None admin] RBAC: Adding query filter params (name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:19.226050 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea389467-1ae3-4924-9d4d-6be4b95bbe3b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:19.227634 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea389467-1ae3-4924-9d4d-6be4b95bbe3b None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:19.256111 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 477/942] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Jun 15 14:32:19 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:19.331211 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:19.344906 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd9494b-a1cb-446b-823e-7b972d32bfbe None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:19.384554 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ec796c-e647-4b8d-8c45-fe853a36e925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:19.385747 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec796c-e647-4b8d-8c45-fe853a36e925 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:19.386508 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec796c-e647-4b8d-8c45-fe853a36e925 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:19.386871 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec796c-e647-4b8d-8c45-fe853a36e925 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:19.395107 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ec796c-e647-4b8d-8c45-fe853a36e925 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:19.396684 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ec796c-e647-4b8d-8c45-fe853a36e925 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:19.413669 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-c1ec796c-e647-4b8d-8c45-fe853a36e925 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 14:32:19.413669 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:19.413669 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:19.413669 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:19.413669 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:19.413669 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:19.413669 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:19.413669 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:19.413669 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:19.413669 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:19.413669 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:19.413669 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:19.413669 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:32:19.413669 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:32:19.413669 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:32:19.413669 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:32:19.414753 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:19.414753 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:19.414753 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:32:19.414753 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:32:19.414753 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:32:19.414753 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:32:19.414753 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:19.414753 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:19.414753 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:19.414753 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:19.414753 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:19.414753 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:19.414753 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:19.414753 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:19.414753 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:19.414753 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:19.414753 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:19.414753 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:19.415679 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:19.415679 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:19.415679 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:32:19.415679 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:32:19.415679 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:32:19.415679 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:32:19.415679 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:32:19.415679 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:32:19.415679 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 14:32:19.415679 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:32:19.415679 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 466/943] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Jun 15 14:32:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:19.423591 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea389467-1ae3-4924-9d4d-6be4b95bbe3b None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:19.472524 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:19.536199 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea389467-1ae3-4924-9d4d-6be4b95bbe3b None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:19.582181 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a010e36-c64f-4101-9989-f22159315cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:19.582883 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a010e36-c64f-4101-9989-f22159315cb0 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:19.583170 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a010e36-c64f-4101-9989-f22159315cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:19.583447 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a010e36-c64f-4101-9989-f22159315cb0 None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:19.584338 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a010e36-c64f-4101-9989-f22159315cb0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:19.584654 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a010e36-c64f-4101-9989-f22159315cb0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:19.586072 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a010e36-c64f-4101-9989-f22159315cb0 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:19.607294 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 478/944] 10.222.0.42 () {56 vars in 1149 bytes} [Wed Jun 15 14:32:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:19.613095 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ec796c-e647-4b8d-8c45-fe853a36e925 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:19.734117 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ec796c-e647-4b8d-8c45-fe853a36e925 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:19.769854 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6332945a-0e8f-4890-b899-07e1784bb2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:19.771901 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6332945a-0e8f-4890-b899-07e1784bb2b7 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:19.771901 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6332945a-0e8f-4890-b899-07e1784bb2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:19.771901 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6332945a-0e8f-4890-b899-07e1784bb2b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:19.782969 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6332945a-0e8f-4890-b899-07e1784bb2b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:19.784431 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6332945a-0e8f-4890-b899-07e1784bb2b7 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:19.803709 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-6332945a-0e8f-4890-b899-07e1784bb2b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:19.803709 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:19.803709 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:19.803709 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:19.803709 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:19.803709 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:19.803709 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:19.803709 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:19.803709 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:19.803709 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:19.803709 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:19.803709 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:19.803709 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:32:19.803709 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:32:19.803709 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:32:19.804678 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:32:19.804678 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:19.804678 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:19.804678 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:19.804678 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:19.804678 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:19.804678 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:19.804678 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:19.804678 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:19.804678 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:19.804678 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:19.804678 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:19.804678 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:19.804678 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:19.804678 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:19.804678 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:19.804678 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:19.805561 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:32:19.805561 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:32:19.805561 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:19.805561 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:32:19.806209 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 467/945] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:32:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:19.806514 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:19.812141 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a010e36-c64f-4101-9989-f22159315cb0 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:19.915558 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f265e-cbb6-414b-859d-66dd47c6f18e None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:19.915920 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f265e-cbb6-414b-859d-66dd47c6f18e None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:19.916122 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f265e-cbb6-414b-859d-66dd47c6f18e None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:19.918814 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 468/946] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:32:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:32:19.924174 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb8b3f6-3302-49f7-8898-1f79e2afc0e0 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:19.924467 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb8b3f6-3302-49f7-8898-1f79e2afc0e0 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:19.924746 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb8b3f6-3302-49f7-8898-1f79e2afc0e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:19.925134 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:19.929191 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a010e36-c64f-4101-9989-f22159315cb0 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:19.948938 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-0fb8b3f6-3302-49f7-8898-1f79e2afc0e0 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:32:19.950399 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-0fb8b3f6-3302-49f7-8898-1f79e2afc0e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:32:19.973559 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a38f26-41de-4989-bf4b-34a3b561964f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:19.974325 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a38f26-41de-4989-bf4b-34a3b561964f None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:19.974662 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a38f26-41de-4989-bf4b-34a3b561964f None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:19.974947 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a38f26-41de-4989-bf4b-34a3b561964f None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:19.980060 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a38f26-41de-4989-bf4b-34a3b561964f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:19.980060 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a38f26-41de-4989-bf4b-34a3b561964f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:19.980060 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a38f26-41de-4989-bf4b-34a3b561964f None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:19.991485 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 479/947] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:32:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:19.997506 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a38f26-41de-4989-bf4b-34a3b561964f None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:20.039489 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 469/948] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:32:19 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:32:20.040842 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64850de-7534-4aa4-b49b-4fcfd0a59669 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:20.041133 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64850de-7534-4aa4-b49b-4fcfd0a59669 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:20.041407 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64850de-7534-4aa4-b49b-4fcfd0a59669 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:20.042775 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 470/949] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:32:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:32:20.047996 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f99ec5-c7c6-4bc5-b4a9-c222de9b1283 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:20.048377 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f99ec5-c7c6-4bc5-b4a9-c222de9b1283 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:20.048668 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f99ec5-c7c6-4bc5-b4a9-c222de9b1283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:20.068937 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-67f99ec5-c7c6-4bc5-b4a9-c222de9b1283 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:32:20.070127 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-67f99ec5-c7c6-4bc5-b4a9-c222de9b1283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:32:20.124834 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a38f26-41de-4989-bf4b-34a3b561964f None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:20.144379 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 471/950] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:32:20 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:32:20.145459 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f99ec5-c7c6-4bc5-b4a9-c222de9b1283 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:20.171336 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177e0161-9e8d-4951-a757-b11d54f0040b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:20.172233 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177e0161-9e8d-4951-a757-b11d54f0040b None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:20.172518 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177e0161-9e8d-4951-a757-b11d54f0040b None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:20.172792 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177e0161-9e8d-4951-a757-b11d54f0040b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:20.179962 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177e0161-9e8d-4951-a757-b11d54f0040b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:20.181428 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177e0161-9e8d-4951-a757-b11d54f0040b None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:20.197344 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-177e0161-9e8d-4951-a757-b11d54f0040b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:32:20.197344 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:20.197344 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:20.197344 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:20.197344 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:20.197344 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:20.197344 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:20.197344 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:20.197344 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:20.197344 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:20.197344 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:20.197344 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:20.197344 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:32:20.197344 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:32:20.197344 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:32:20.198289 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:32:20.198289 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:20.198289 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:20.198289 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:20.198289 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:20.198289 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:20.198289 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:20.198289 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:20.198289 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:20.198289 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:20.198289 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:20.198289 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:20.198289 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:20.198289 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:20.198289 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:20.198289 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:20.198289 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:20.199235 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:32:20.199235 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:32:20.199235 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:32:20.199235 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:32:20.199235 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:32:20.199235 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:32:20.199235 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:32:20.199235 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:20.199235 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 480/951] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jun 15 14:32:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:20.199827 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177e0161-9e8d-4951-a757-b11d54f0040b None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:20.213062 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:20.220480 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:20.255975 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f99ec5-c7c6-4bc5-b4a9-c222de9b1283 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:20.294007 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4445c2c-de11-467b-9084-0732e0ac8ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:20.294806 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4445c2c-de11-467b-9084-0732e0ac8ba3 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:20.295090 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4445c2c-de11-467b-9084-0732e0ac8ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:20.295369 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4445c2c-de11-467b-9084-0732e0ac8ba3 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:20.296016 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4445c2c-de11-467b-9084-0732e0ac8ba3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:20.297423 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4445c2c-de11-467b-9084-0732e0ac8ba3 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:20.303429 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177e0161-9e8d-4951-a757-b11d54f0040b None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:20.309758 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-f4445c2c-de11-467b-9084-0732e0ac8ba3 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 14:32:20.309758 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:20.309758 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:32:20.309758 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:32:20.309758 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:32:20.309758 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:32:20.309758 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:32:20.309758 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:32:20.309758 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 14:32:20.309758 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:32:20.309758 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:32:20.309758 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:32:20.309758 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:20.309758 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:20.309758 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:20.309758 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:20.309758 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:20.312297 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:20.312297 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:20.312297 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:20.312297 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:20.312297 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:20.312297 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:20.312297 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:32:20.312297 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:32:20.312297 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:32:20.312297 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:32:20.312297 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:20.312297 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:20.312297 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:20.312297 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:20.312297 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:20.312297 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:20.313264 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:20.313264 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:20.313264 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:20.313264 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:20.313264 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:20.313264 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:20.313264 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:20.313264 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:20.313264 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:20.313264 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:20.313264 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:32:20.313264 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:32:20.313264 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 14:32:20.313264 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:32:20.313264 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 472/952] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jun 15 14:32:20 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:20.314161 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4445c2c-de11-467b-9084-0732e0ac8ba3 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:20.344190 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107505a1-b964-4d66-af04-81ee9a1a5dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:20.344990 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107505a1-b964-4d66-af04-81ee9a1a5dfa None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:20.345270 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107505a1-b964-4d66-af04-81ee9a1a5dfa None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:20.345541 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107505a1-b964-4d66-af04-81ee9a1a5dfa None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:20.346189 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107505a1-b964-4d66-af04-81ee9a1a5dfa None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:20.347536 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107505a1-b964-4d66-af04-81ee9a1a5dfa None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:20.359955 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-107505a1-b964-4d66-af04-81ee9a1a5dfa None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 14:32:20.359955 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:20.359955 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:32:20.359955 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:32:20.359955 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:32:20.359955 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:32:20.359955 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:32:20.359955 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:32:20.359955 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 14:32:20.359955 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:20.359955 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:32:20.359955 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:20.359955 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:20.359955 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:20.359955 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:20.359955 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:20.359955 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:20.361326 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:20.361326 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:20.361326 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:20.361326 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:20.361326 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:20.361326 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:20.361326 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:32:20.361326 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:32:20.361326 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:32:20.361326 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:32:20.361326 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:20.361326 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:20.361326 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:20.361326 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:20.361326 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:20.361326 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:20.362673 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:20.362673 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:20.362673 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:20.362673 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:20.362673 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:20.362673 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:20.362673 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:20.362673 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:20.362673 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:20.362673 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:20.362673 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:32:20.362673 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:32:20.362673 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 14:32:20.362673 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:20.362673 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 481/953] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Jun 15 14:32:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:20.363488 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107505a1-b964-4d66-af04-81ee9a1a5dfa None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:20.427042 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4445c2c-de11-467b-9084-0732e0ac8ba3 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:20.445851 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:20.450555 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:20.460758 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ee4f23-dee1-46df-a575-71cc6deac868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:20.461417 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ee4f23-dee1-46df-a575-71cc6deac868 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:20.461699 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ee4f23-dee1-46df-a575-71cc6deac868 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:20.461971 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ee4f23-dee1-46df-a575-71cc6deac868 None admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:20.462757 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ee4f23-dee1-46df-a575-71cc6deac868 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:20.463069 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ee4f23-dee1-46df-a575-71cc6deac868 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:20.465279 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ee4f23-dee1-46df-a575-71cc6deac868 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:20.466489 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107505a1-b964-4d66-af04-81ee9a1a5dfa None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:20.483026 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 473/954] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jun 15 14:32:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:20.483026 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ee4f23-dee1-46df-a575-71cc6deac868 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:20.508235 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1931e125-f230-41e4-9c3c-aafb9afef947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:20.508981 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931e125-f230-41e4-9c3c-aafb9afef947 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:20.509185 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931e125-f230-41e4-9c3c-aafb9afef947 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:20.509446 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931e125-f230-41e4-9c3c-aafb9afef947 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:20.510149 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1931e125-f230-41e4-9c3c-aafb9afef947 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:20.510484 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1931e125-f230-41e4-9c3c-aafb9afef947 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:20.512426 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1931e125-f230-41e4-9c3c-aafb9afef947 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:20.522263 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 482/955] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Jun 15 14:32:20 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:20.522263 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1931e125-f230-41e4-9c3c-aafb9afef947 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:20.597085 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ee4f23-dee1-46df-a575-71cc6deac868 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:20.635884 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0df358-00cd-4fbd-b7a3-0bd62af28b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:20.637527 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0df358-00cd-4fbd-b7a3-0bd62af28b39 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:20.637860 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0df358-00cd-4fbd-b7a3-0bd62af28b39 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:20.638144 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0df358-00cd-4fbd-b7a3-0bd62af28b39 None admin] PATH_INFO: `/v3/services` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:20.638859 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0df358-00cd-4fbd-b7a3-0bd62af28b39 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:20.639177 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0df358-00cd-4fbd-b7a3-0bd62af28b39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:20.640611 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0df358-00cd-4fbd-b7a3-0bd62af28b39 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:20.643092 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1931e125-f230-41e4-9c3c-aafb9afef947 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:20.650223 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 474/956] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Jun 15 14:32:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:20.650223 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0df358-00cd-4fbd-b7a3-0bd62af28b39 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:20.684696 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e501f845-fedf-4765-8ff5-5885c51fc7f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:20.685822 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501f845-fedf-4765-8ff5-5885c51fc7f5 None admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:20.686275 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501f845-fedf-4765-8ff5-5885c51fc7f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:20.686705 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501f845-fedf-4765-8ff5-5885c51fc7f5 None admin] PATH_INFO: `/v3/projects/3094ff8966f149bda784da6a51b1358f/users/19d82c04f9994654adfb10f4bd0573c2/roles/76f33d89f76347be9ef2fdac0435173f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:20.689665 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e501f845-fedf-4765-8ff5-5885c51fc7f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=3094ff8966f149bda784da6a51b1358f, user_id=19d82c04f9994654adfb10f4bd0573c2, role_id=76f33d89f76347be9ef2fdac0435173f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:20.691983 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e501f845-fedf-4765-8ff5-5885c51fc7f5 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:20.697779 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:20.747959 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0df358-00cd-4fbd-b7a3-0bd62af28b39 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:20.765425 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 483/957] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Jun 15 14:32:20 2022] PUT /identity/v3/projects/3094ff8966f149bda784da6a51b1358f/users/19d82c04f9994654adfb10f4bd0573c2/roles/76f33d89f76347be9ef2fdac0435173f => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:32:20.766441 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e501f845-fedf-4765-8ff5-5885c51fc7f5 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:20.784840 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:20.796074 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36bd0c6-a5e7-4435-8cd2-ea723d30e24c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:20.797639 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36bd0c6-a5e7-4435-8cd2-ea723d30e24c None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:20.797989 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36bd0c6-a5e7-4435-8cd2-ea723d30e24c None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:20.798338 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36bd0c6-a5e7-4435-8cd2-ea723d30e24c None admin] PATH_INFO: `/v3/services` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:20.799067 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36bd0c6-a5e7-4435-8cd2-ea723d30e24c None admin] RBAC: Adding query filter params (type=compute) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:20.799375 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36bd0c6-a5e7-4435-8cd2-ea723d30e24c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:20.800876 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36bd0c6-a5e7-4435-8cd2-ea723d30e24c None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:20.806853 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 475/958] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Jun 15 14:32:20 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:20.911283 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e501f845-fedf-4765-8ff5-5885c51fc7f5 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:20.952998 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460a2a61-f9d7-4750-8091-bfdd708fe6e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:20.953686 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a2a61-f9d7-4750-8091-bfdd708fe6e7 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:20.953968 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a2a61-f9d7-4750-8091-bfdd708fe6e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:20.954250 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a2a61-f9d7-4750-8091-bfdd708fe6e7 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:20.954976 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460a2a61-f9d7-4750-8091-bfdd708fe6e7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:20.955285 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460a2a61-f9d7-4750-8091-bfdd708fe6e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:20.956773 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460a2a61-f9d7-4750-8091-bfdd708fe6e7 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:20.962984 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 484/959] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Jun 15 14:32:20 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:20.969663 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36bd0c6-a5e7-4435-8cd2-ea723d30e24c None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:21.088773 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36bd0c6-a5e7-4435-8cd2-ea723d30e24c None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:21.102451 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:21.130095 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5716fa9-e615-4a4c-a749-efcb270b571b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:21.131002 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5716fa9-e615-4a4c-a749-efcb270b571b None admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:21.131431 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5716fa9-e615-4a4c-a749-efcb270b571b None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:21.131850 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5716fa9-e615-4a4c-a749-efcb270b571b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:21.132848 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5716fa9-e615-4a4c-a749-efcb270b571b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:21.134373 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5716fa9-e615-4a4c-a749-efcb270b571b None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:21.223000 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 476/960] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Jun 15 14:32:20 2022] POST /identity/v3/endpoints => generated 349 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:32:21.628460 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ad0528-e556-4bd4-87df-25fc21e0b2a0 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:21.629413 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ad0528-e556-4bd4-87df-25fc21e0b2a0 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:21.629933 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ad0528-e556-4bd4-87df-25fc21e0b2a0 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:21.631771 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 485/961] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:32:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:32:21.637644 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1425114e-ec7b-4c3b-a9d6-8530fc94f25e None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:21.637937 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1425114e-ec7b-4c3b-a9d6-8530fc94f25e None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:21.638223 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1425114e-ec7b-4c3b-a9d6-8530fc94f25e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:21.656268 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-1425114e-ec7b-4c3b-a9d6-8530fc94f25e None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:32:21.657483 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-1425114e-ec7b-4c3b-a9d6-8530fc94f25e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:32:21.733778 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 477/962] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:32:21 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:32:21.758373 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ad0528-e556-4bd4-87df-25fc21e0b2a0 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:21.830826 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:21.862471 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ad0528-e556-4bd4-87df-25fc21e0b2a0 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:21.896608 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6465a6c6-ded5-4cfe-b349-53507cb743eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:21.897271 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6465a6c6-ded5-4cfe-b349-53507cb743eb None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:21.897551 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6465a6c6-ded5-4cfe-b349-53507cb743eb None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:21.897827 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6465a6c6-ded5-4cfe-b349-53507cb743eb None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:21.898533 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6465a6c6-ded5-4cfe-b349-53507cb743eb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:21.900461 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6465a6c6-ded5-4cfe-b349-53507cb743eb None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:21.935449 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.sql.core [None req-6465a6c6-ded5-4cfe-b349-53507cb743eb None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 15 14:32:21.935449 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 15 14:32:21.935449 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [parameters: {'user_id': 'dc06e8c3110e45b0956de6dfac49b17c', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 15 14:32:21.935449 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80600) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 15 14:32:21.943093 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-6465a6c6-ded5-4cfe-b349-53507cb743eb None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 15 14:32:21.943093 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:21.943093 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 15 14:32:21.943093 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 15 14:32:21.943093 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 15 14:32:21.943093 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 15 14:32:21.943093 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 15 14:32:21.943093 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application result = self._query(query) Jun 15 14:32:21.943093 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 15 14:32:21.943093 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application conn.query(q) Jun 15 14:32:21.943093 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 15 14:32:21.943093 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 15 14:32:21.943093 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 15 14:32:21.943093 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application result.read() Jun 15 14:32:21.943093 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 15 14:32:21.943986 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 15 14:32:21.943986 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 15 14:32:21.943986 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 15 14:32:21.943986 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 15 14:32:21.943986 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 15 14:32:21.943986 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 15 14:32:21.943986 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 15 14:32:21.943986 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 15 14:32:21.943986 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:21.943986 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 15 14:32:21.943986 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:21.943986 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:21.943986 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 15 14:32:21.943986 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 15 14:32:21.943986 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 15 14:32:21.943986 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 15 14:32:21.943986 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 15 14:32:21.943986 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application next(self.gen) Jun 15 14:32:21.945150 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 15 14:32:21.945150 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application yield resource Jun 15 14:32:21.945150 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 15 14:32:21.945150 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application next(self.gen) Jun 15 14:32:21.945150 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 15 14:32:21.945150 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self.session.rollback() Jun 15 14:32:21.945150 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 15 14:32:21.945150 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self.force_reraise() Jun 15 14:32:21.945150 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 15 14:32:21.945150 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise self.value Jun 15 14:32:21.945150 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 15 14:32:21.945150 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 15 14:32:21.945150 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 15 14:32:21.945150 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application session.commit() Jun 15 14:32:21.945150 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 15 14:32:21.945150 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 15 14:32:21.945150 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 15 14:32:21.946280 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._prepare_impl() Jun 15 14:32:21.946280 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 15 14:32:21.946280 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self.session.flush() Jun 15 14:32:21.946280 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 15 14:32:21.946280 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._flush(objects) Jun 15 14:32:21.946280 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 15 14:32:21.946280 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 15 14:32:21.946280 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 15 14:32:21.946280 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application compat.raise_( Jun 15 14:32:21.946280 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 15 14:32:21.946280 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception Jun 15 14:32:21.946280 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 15 14:32:21.946280 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application flush_context.execute() Jun 15 14:32:21.946280 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 15 14:32:21.946280 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rec.execute(self) Jun 15 14:32:21.946280 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 15 14:32:21.946280 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 15 14:32:21.947154 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 15 14:32:21.947154 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 15 14:32:21.947154 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 15 14:32:21.947154 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 15 14:32:21.947154 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 15 14:32:21.947154 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 15 14:32:21.947154 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 15 14:32:21.947154 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 15 14:32:21.947154 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 15 14:32:21.947154 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 15 14:32:21.947154 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 15 14:32:21.947154 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 15 14:32:21.947154 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 15 14:32:21.947154 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 15 14:32:21.947154 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 15 14:32:21.947154 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception Jun 15 14:32:21.948007 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 15 14:32:21.948007 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 15 14:32:21.948007 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 15 14:32:21.948007 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 15 14:32:21.948007 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 15 14:32:21.948007 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application result = self._query(query) Jun 15 14:32:21.948007 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 15 14:32:21.948007 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application conn.query(q) Jun 15 14:32:21.948007 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 15 14:32:21.948007 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 15 14:32:21.948007 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 15 14:32:21.948007 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application result.read() Jun 15 14:32:21.948007 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 15 14:32:21.948007 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 15 14:32:21.948007 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 15 14:32:21.948007 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 15 14:32:21.964562 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 15 14:32:21.964562 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 15 14:32:21.964562 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 15 14:32:21.964562 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 15 14:32:21.964562 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 15 14:32:21.964562 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 15 14:32:21.964562 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [parameters: {'user_id': 'dc06e8c3110e45b0956de6dfac49b17c', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 15 14:32:21.964562 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 15 14:32:21.964562 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:21.964562 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:32:21.964562 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:21.964562 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:21.964562 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:21.964562 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:21.964562 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:21.966476 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:21.966476 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:21.966476 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:21.966476 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:21.966476 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:21.966476 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:21.966476 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:21.966476 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 15 14:32:21.966476 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 15 14:32:21.966476 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:21.966476 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:21.966476 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:32:21.966476 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:32:21.966476 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:32:21.966476 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:32:21.966476 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 15 14:32:21.966476 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 15 14:32:21.968629 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 15 14:32:21.968629 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 15 14:32:21.968629 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 15 14:32:21.968629 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 15 14:32:21.968629 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 15 14:32:21.968629 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:21.968629 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 486/963] 10.222.0.42 () {60 vars in 1131 bytes} [Wed Jun 15 14:32:21 2022] POST /identity/v3/users => generated 160 bytes in 187 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:21.969533 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1425114e-ec7b-4c3b-a9d6-8530fc94f25e None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:22.057475 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:22.065206 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1425114e-ec7b-4c3b-a9d6-8530fc94f25e None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:22.102786 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c501f4c8-f2d3-4f95-bc9f-c47b1ad7eeab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:22.103555 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c501f4c8-f2d3-4f95-bc9f-c47b1ad7eeab None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:22.103830 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c501f4c8-f2d3-4f95-bc9f-c47b1ad7eeab None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:22.104205 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c501f4c8-f2d3-4f95-bc9f-c47b1ad7eeab None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:22.111919 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c501f4c8-f2d3-4f95-bc9f-c47b1ad7eeab None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:22.113364 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c501f4c8-f2d3-4f95-bc9f-c47b1ad7eeab None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:22.128816 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-c501f4c8-f2d3-4f95-bc9f-c47b1ad7eeab None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 14:32:22.128816 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:22.128816 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:22.128816 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:22.128816 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:22.128816 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:22.128816 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:22.128816 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:22.128816 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:22.128816 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:22.128816 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:22.128816 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:22.128816 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:32:22.128816 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:32:22.128816 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:32:22.128816 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:32:22.131016 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:22.131016 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:22.131016 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:32:22.131016 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:32:22.131016 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:32:22.131016 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:32:22.131016 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:22.131016 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:22.131016 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:22.131016 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:22.131016 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:22.131016 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:22.131016 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:22.131016 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:22.131016 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:22.131016 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:22.131016 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:22.131016 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:22.133415 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:22.133415 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:22.133415 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:32:22.133415 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:32:22.133415 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:32:22.133415 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:32:22.133415 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:32:22.133415 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:32:22.133415 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 14:32:22.133415 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:32:22.133415 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 478/964] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jun 15 14:32:21 2022] GET /identity/v3/users/cinder => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:22.137348 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6465a6c6-ded5-4cfe-b349-53507cb743eb None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:22.243712 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6465a6c6-ded5-4cfe-b349-53507cb743eb None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:22.273540 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:22.278781 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a271b25-56e8-47f6-96ce-6161c9b8f820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:22.279414 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a271b25-56e8-47f6-96ce-6161c9b8f820 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:22.279692 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a271b25-56e8-47f6-96ce-6161c9b8f820 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:22.279959 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a271b25-56e8-47f6-96ce-6161c9b8f820 None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:22.280861 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a271b25-56e8-47f6-96ce-6161c9b8f820 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:22.281164 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a271b25-56e8-47f6-96ce-6161c9b8f820 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:22.282745 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a271b25-56e8-47f6-96ce-6161c9b8f820 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:22.321361 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 487/965] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Jun 15 14:32:22 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:23.022200 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc203cae-b8c1-4896-a37b-628ae637efda None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:23.022719 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc203cae-b8c1-4896-a37b-628ae637efda None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:23.022835 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc203cae-b8c1-4896-a37b-628ae637efda None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:23.024681 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 479/966] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:32:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:32:23.031022 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9860c9d0-184c-46ee-b87e-8ae811595ce2 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:23.031562 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9860c9d0-184c-46ee-b87e-8ae811595ce2 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:23.031562 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9860c9d0-184c-46ee-b87e-8ae811595ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:23.051140 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-9860c9d0-184c-46ee-b87e-8ae811595ce2 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:32:23.052927 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-9860c9d0-184c-46ee-b87e-8ae811595ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:32:23.115025 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e7d93a-6111-4cc2-8e3d-b4abc1b4e8b4 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:23.115314 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e7d93a-6111-4cc2-8e3d-b4abc1b4e8b4 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:23.115736 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e7d93a-6111-4cc2-8e3d-b4abc1b4e8b4 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:23.117271 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 480/967] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:32:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:32:23.122827 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04982314-7e65-4f0c-abdf-4b08fc847d69 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:23.123115 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04982314-7e65-4f0c-abdf-4b08fc847d69 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:23.123507 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04982314-7e65-4f0c-abdf-4b08fc847d69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:23.132229 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 488/968] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:32:23 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:32:23.140021 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-04982314-7e65-4f0c-abdf-4b08fc847d69 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:32:23.141224 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-04982314-7e65-4f0c-abdf-4b08fc847d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:32:23.151446 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9860c9d0-184c-46ee-b87e-8ae811595ce2 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:23.228262 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 481/969] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:32:23 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:32:23.242481 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04982314-7e65-4f0c-abdf-4b08fc847d69 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:23.255163 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9860c9d0-184c-46ee-b87e-8ae811595ce2 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:23.277111 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:23.293794 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b231e8c-f648-4943-b4b1-189acb90d748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:23.294813 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b231e8c-f648-4943-b4b1-189acb90d748 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:23.295256 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b231e8c-f648-4943-b4b1-189acb90d748 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:23.295530 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b231e8c-f648-4943-b4b1-189acb90d748 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:23.306566 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b231e8c-f648-4943-b4b1-189acb90d748 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:32:23.315888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-2b231e8c-f648-4943-b4b1-189acb90d748 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 14:32:23.315888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:23.315888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:23.315888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:23.315888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:23.315888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:23.315888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:23.315888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:23.315888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:23.315888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:23.315888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:23.315888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:23.315888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:32:23.315888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:32:23.315888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:32:23.315888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:32:23.317146 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:32:23.317146 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:32:23.317146 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:32:23.317146 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:32:23.317146 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:23.317146 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:23.317146 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:23.317146 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:23.317146 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:23.317146 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:23.317146 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:23.317146 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:23.317146 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:23.317146 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:23.317146 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:23.317146 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:23.318007 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:23.318007 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:23.318007 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:23.318007 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:23.318007 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:32:23.318007 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:32:23.318007 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:32:23.318007 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:32:23.318007 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:32:23.318007 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:32:23.318007 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 14:32:23.318007 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:23.322414 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 489/970] 10.222.0.42 () {56 vars in 1120 bytes} [Wed Jun 15 14:32:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:32:23.323248 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b231e8c-f648-4943-b4b1-189acb90d748 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:23.334283 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:23.352004 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04982314-7e65-4f0c-abdf-4b08fc847d69 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:23.389919 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362bfd9b-c7f6-4d5f-8585-e7aba64e36ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:23.390594 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362bfd9b-c7f6-4d5f-8585-e7aba64e36ce None admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:23.390878 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362bfd9b-c7f6-4d5f-8585-e7aba64e36ce None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:23.391151 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362bfd9b-c7f6-4d5f-8585-e7aba64e36ce None admin] PATH_INFO: `/v3/services` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:23.391895 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362bfd9b-c7f6-4d5f-8585-e7aba64e36ce None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:23.393361 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362bfd9b-c7f6-4d5f-8585-e7aba64e36ce None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:23.433611 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b231e8c-f648-4943-b4b1-189acb90d748 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:23.466583 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b62c31-0297-4885-8a96-d926e13f4144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:23.467242 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b62c31-0297-4885-8a96-d926e13f4144 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:23.467524 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b62c31-0297-4885-8a96-d926e13f4144 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:23.467800 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b62c31-0297-4885-8a96-d926e13f4144 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:23.468592 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b62c31-0297-4885-8a96-d926e13f4144 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:23.468897 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b62c31-0297-4885-8a96-d926e13f4144 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:23.470234 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b62c31-0297-4885-8a96-d926e13f4144 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:23.478119 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 490/971] 10.222.0.42 () {56 vars in 1115 bytes} [Wed Jun 15 14:32:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:32:23.479290 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5099486-77d6-4a93-b82d-2bf9202d5785 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:23.479573 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5099486-77d6-4a93-b82d-2bf9202d5785 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:23.480582 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5099486-77d6-4a93-b82d-2bf9202d5785 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:23.481930 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 491/972] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:32:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:32:23.484434 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5099486-77d6-4a93-b82d-2bf9202d5785 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:23.504115 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 482/973] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Jun 15 14:32:23 2022] POST /identity/v3/services => generated 242 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:32:23.504115 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ffd4d2-e708-4047-970f-c7ebf9d60f21 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:23.504115 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ffd4d2-e708-4047-970f-c7ebf9d60f21 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:23.504115 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ffd4d2-e708-4047-970f-c7ebf9d60f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:23.511100 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-96ffd4d2-e708-4047-970f-c7ebf9d60f21 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:32:23.512574 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-96ffd4d2-e708-4047-970f-c7ebf9d60f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:32:23.545483 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:23.600987 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 483/974] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:32:23 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:32:23.614245 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5099486-77d6-4a93-b82d-2bf9202d5785 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:23.624964 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ffd4d2-e708-4047-970f-c7ebf9d60f21 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:23.654939 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7c48c1-0335-4a62-a825-1dd4df6c12d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:23.655738 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c48c1-0335-4a62-a825-1dd4df6c12d8 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:23.656024 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c48c1-0335-4a62-a825-1dd4df6c12d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:23.656332 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c48c1-0335-4a62-a825-1dd4df6c12d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:23.670776 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7c48c1-0335-4a62-a825-1dd4df6c12d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:23.672783 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7c48c1-0335-4a62-a825-1dd4df6c12d8 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:23.694457 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-6f7c48c1-0335-4a62-a825-1dd4df6c12d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:23.694457 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:23.694457 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:23.694457 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:23.694457 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:23.694457 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:23.694457 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:23.694457 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:23.694457 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:23.694457 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:23.694457 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:23.694457 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:23.694457 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:32:23.694457 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:32:23.694457 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:32:23.695433 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:32:23.695433 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:23.695433 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:23.695433 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:23.695433 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:23.695433 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:23.695433 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:23.695433 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:23.695433 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:23.695433 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:23.695433 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:23.695433 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:23.695433 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:23.695433 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:23.695433 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:23.695433 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:23.695433 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:23.696447 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:32:23.696447 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:32:23.696447 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:23.696447 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:23.696447 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 492/975] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:32:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:32:23.702348 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7c48c1-0335-4a62-a825-1dd4df6c12d8 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:23.704801 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:23.739093 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ffd4d2-e708-4047-970f-c7ebf9d60f21 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:23.777368 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041cb7d9-1607-4e88-ba1c-207cf46036f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:23.778275 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041cb7d9-1607-4e88-ba1c-207cf46036f4 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:23.778562 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041cb7d9-1607-4e88-ba1c-207cf46036f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:23.778838 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041cb7d9-1607-4e88-ba1c-207cf46036f4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:23.779492 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041cb7d9-1607-4e88-ba1c-207cf46036f4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:23.780889 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041cb7d9-1607-4e88-ba1c-207cf46036f4 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:23.793269 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-041cb7d9-1607-4e88-ba1c-207cf46036f4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 14:32:23.793269 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:23.793269 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:32:23.793269 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:32:23.793269 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:32:23.793269 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:32:23.793269 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:32:23.793269 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:32:23.793269 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 14:32:23.793269 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:32:23.793269 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:32:23.793269 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:32:23.793269 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:23.793269 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:23.793269 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:23.793269 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:23.793269 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:23.794325 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:23.794325 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:23.794325 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:23.794325 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:23.794325 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:23.794325 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:23.794325 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:32:23.794325 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:32:23.794325 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:32:23.794325 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:32:23.794325 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:23.794325 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:23.794325 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:23.794325 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:23.794325 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:23.794325 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:23.795862 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:23.795862 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:23.795862 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:23.795862 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:23.795862 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:23.795862 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:23.795862 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:23.795862 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:23.795862 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:23.795862 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:23.795862 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:32:23.795862 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:32:23.795862 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 14:32:23.795862 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:32:23.795862 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 484/976] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Jun 15 14:32:23 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:23.800244 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041cb7d9-1607-4e88-ba1c-207cf46036f4 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:23.802901 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:23.810673 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7c48c1-0335-4a62-a825-1dd4df6c12d8 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:23.848886 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ea276b-4d7a-4821-ba06-e797c95f820d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:23.849788 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ea276b-4d7a-4821-ba06-e797c95f820d None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:23.850281 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ea276b-4d7a-4821-ba06-e797c95f820d None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:23.850859 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ea276b-4d7a-4821-ba06-e797c95f820d None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:23.851613 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ea276b-4d7a-4821-ba06-e797c95f820d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:23.851925 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ea276b-4d7a-4821-ba06-e797c95f820d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:23.853315 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ea276b-4d7a-4821-ba06-e797c95f820d None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:23.871893 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 493/977] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:32:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:32:23.874076 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ea276b-4d7a-4821-ba06-e797c95f820d None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:23.906740 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041cb7d9-1607-4e88-ba1c-207cf46036f4 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:23.941592 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027b2adc-9cd3-454d-874e-c1d3bfd79066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:23.942252 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027b2adc-9cd3-454d-874e-c1d3bfd79066 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:23.942532 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027b2adc-9cd3-454d-874e-c1d3bfd79066 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:23.943799 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027b2adc-9cd3-454d-874e-c1d3bfd79066 None admin] PATH_INFO: `/v3/services` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:23.943799 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027b2adc-9cd3-454d-874e-c1d3bfd79066 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:23.943971 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027b2adc-9cd3-454d-874e-c1d3bfd79066 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:23.945211 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027b2adc-9cd3-454d-874e-c1d3bfd79066 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:23.946284 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:23.950861 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 485/978] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Jun 15 14:32:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:32:23.956218 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027b2adc-9cd3-454d-874e-c1d3bfd79066 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:23.982513 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ea276b-4d7a-4821-ba06-e797c95f820d None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:24.022992 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5de832-e70c-435d-9a3e-112ad538661b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:24.024054 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5de832-e70c-435d-9a3e-112ad538661b None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:24.024360 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5de832-e70c-435d-9a3e-112ad538661b None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:24.024636 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5de832-e70c-435d-9a3e-112ad538661b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:24.033101 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5de832-e70c-435d-9a3e-112ad538661b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:24.034533 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5de832-e70c-435d-9a3e-112ad538661b None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:24.051371 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-5f5de832-e70c-435d-9a3e-112ad538661b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 14:32:24.051371 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:24.051371 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:24.051371 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:24.051371 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:24.051371 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:24.051371 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:24.051371 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:24.051371 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:24.051371 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:24.051371 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:24.051371 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:24.051371 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:32:24.051371 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:32:24.051371 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:32:24.051371 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:32:24.052383 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:24.052383 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:24.052383 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:32:24.052383 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:32:24.052383 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:32:24.052383 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:32:24.052383 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:24.052383 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:24.052383 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:24.052383 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:24.052383 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:24.052383 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:24.052383 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:24.052383 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:24.052383 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:24.052383 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:24.052383 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:24.052383 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:24.053357 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:24.053357 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:24.053357 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:32:24.053357 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:32:24.053357 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:32:24.053357 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:32:24.053357 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:32:24.053357 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:32:24.053357 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 14:32:24.053357 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:24.053357 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 494/979] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Jun 15 14:32:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:32:24.059069 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5de832-e70c-435d-9a3e-112ad538661b None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:24.068222 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027b2adc-9cd3-454d-874e-c1d3bfd79066 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:24.076397 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:24.105614 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c3f013-9477-424f-8617-981c2da3db08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:24.106565 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c3f013-9477-424f-8617-981c2da3db08 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:24.106844 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c3f013-9477-424f-8617-981c2da3db08 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:24.107129 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c3f013-9477-424f-8617-981c2da3db08 None admin] PATH_INFO: `/v3/services` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:24.107827 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c3f013-9477-424f-8617-981c2da3db08 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:24.108159 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c3f013-9477-424f-8617-981c2da3db08 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:24.109524 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c3f013-9477-424f-8617-981c2da3db08 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:24.114431 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 486/980] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Jun 15 14:32:23 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:32:24.121196 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c3f013-9477-424f-8617-981c2da3db08 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:24.163877 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5de832-e70c-435d-9a3e-112ad538661b None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:24.187537 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:24.199333 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb7d111-ad22-41db-9c77-ecd2ce1c775a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:24.200005 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb7d111-ad22-41db-9c77-ecd2ce1c775a None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:24.200317 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb7d111-ad22-41db-9c77-ecd2ce1c775a None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:24.203048 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb7d111-ad22-41db-9c77-ecd2ce1c775a None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:24.203048 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb7d111-ad22-41db-9c77-ecd2ce1c775a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:24.203048 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb7d111-ad22-41db-9c77-ecd2ce1c775a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:24.203402 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb7d111-ad22-41db-9c77-ecd2ce1c775a None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:24.221757 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 495/981] 10.222.0.42 () {56 vars in 1149 bytes} [Wed Jun 15 14:32:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:32:24.227273 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb7d111-ad22-41db-9c77-ecd2ce1c775a None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:24.235034 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c3f013-9477-424f-8617-981c2da3db08 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:24.269776 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63a9898-8c21-4b64-a3fa-7774ca86298d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:24.270434 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63a9898-8c21-4b64-a3fa-7774ca86298d None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:24.270716 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63a9898-8c21-4b64-a3fa-7774ca86298d None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:24.271011 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63a9898-8c21-4b64-a3fa-7774ca86298d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:24.271788 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63a9898-8c21-4b64-a3fa-7774ca86298d None admin] RBAC: Adding query filter params (interface=admin, service_id=b80b04e510564033a7f22aa02a862777, region_id=RegionOne) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:24.272116 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63a9898-8c21-4b64-a3fa-7774ca86298d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:24.273482 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63a9898-8c21-4b64-a3fa-7774ca86298d None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:24.283900 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 487/982] 10.222.0.42 () {56 vars in 1249 bytes} [Wed Jun 15 14:32:24 2022] GET /identity/v3/endpoints?service_id=b80b04e510564033a7f22aa02a862777&interface=admin®ion_id=RegionOne => generated 193 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:32:24.285143 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63a9898-8c21-4b64-a3fa-7774ca86298d None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:24.305975 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:24.339806 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb7d111-ad22-41db-9c77-ecd2ce1c775a None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:24.379618 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdd4c03-668d-47cc-9d1f-c387700f1fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:24.380307 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdd4c03-668d-47cc-9d1f-c387700f1fcd None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:24.380596 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdd4c03-668d-47cc-9d1f-c387700f1fcd None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:24.380876 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdd4c03-668d-47cc-9d1f-c387700f1fcd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:24.392406 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63a9898-8c21-4b64-a3fa-7774ca86298d None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:24.425514 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:24.435466 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d331a659-bd57-4329-8d7e-603e0905ed01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:24.437198 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d331a659-bd57-4329-8d7e-603e0905ed01 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:24.437198 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d331a659-bd57-4329-8d7e-603e0905ed01 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:24.437198 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d331a659-bd57-4329-8d7e-603e0905ed01 None admin] PATH_INFO: `/v3/services` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:24.437446 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d331a659-bd57-4329-8d7e-603e0905ed01 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:24.438712 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d331a659-bd57-4329-8d7e-603e0905ed01 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:24.456121 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 488/983] 10.222.0.42 () {56 vars in 1087 bytes} [Wed Jun 15 14:32:24 2022] GET /identity/v3/services => generated 1727 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:32:24.463625 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdd4c03-668d-47cc-9d1f-c387700f1fcd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:24.465491 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdd4c03-668d-47cc-9d1f-c387700f1fcd None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:24.544940 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:24.583244 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 496/984] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jun 15 14:32:24 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 15 14:32:24.584500 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d331a659-bd57-4329-8d7e-603e0905ed01 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:24.693065 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d331a659-bd57-4329-8d7e-603e0905ed01 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:24.727316 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a24c07-43ab-4bc9-b7e3-8975914afad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:24.728121 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a24c07-43ab-4bc9-b7e3-8975914afad8 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:24.728405 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a24c07-43ab-4bc9-b7e3-8975914afad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:24.728674 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a24c07-43ab-4bc9-b7e3-8975914afad8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:24.739797 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a24c07-43ab-4bc9-b7e3-8975914afad8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:24.741257 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a24c07-43ab-4bc9-b7e3-8975914afad8 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:24.758886 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-b7a24c07-43ab-4bc9-b7e3-8975914afad8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:24.758886 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:24.758886 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:24.758886 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:24.758886 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:24.758886 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:24.758886 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:24.758886 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:24.758886 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:24.758886 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:24.758886 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:24.758886 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:24.758886 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:32:24.758886 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:32:24.758886 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:32:24.759807 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:32:24.759807 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:24.759807 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:24.759807 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:24.759807 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:24.759807 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:24.759807 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:24.759807 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:24.759807 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:24.759807 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:24.759807 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:24.759807 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:24.759807 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:24.759807 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:24.759807 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:24.759807 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:24.759807 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:24.760780 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:32:24.760780 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:32:24.760780 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:24.760780 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:32:24.764398 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 489/985] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:32:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:24.765786 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdd4c03-668d-47cc-9d1f-c387700f1fcd None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:24.868235 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdd4c03-668d-47cc-9d1f-c387700f1fcd None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:24.903029 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac254267-6aa9-44f9-aa98-2c7a16ca596f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:24.903834 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac254267-6aa9-44f9-aa98-2c7a16ca596f None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:24.904256 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac254267-6aa9-44f9-aa98-2c7a16ca596f None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:24.904638 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac254267-6aa9-44f9-aa98-2c7a16ca596f None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:24.905460 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac254267-6aa9-44f9-aa98-2c7a16ca596f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:24.905901 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac254267-6aa9-44f9-aa98-2c7a16ca596f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:24.907357 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac254267-6aa9-44f9-aa98-2c7a16ca596f None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:24.924183 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 497/986] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:32:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:24.924700 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a24c07-43ab-4bc9-b7e3-8975914afad8 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:24.970241 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:25.030256 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a24c07-43ab-4bc9-b7e3-8975914afad8 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:25.064445 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ccd202-4630-40f3-b2b2-c1669b2c4abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:25.065237 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ccd202-4630-40f3-b2b2-c1669b2c4abc None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:25.065522 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ccd202-4630-40f3-b2b2-c1669b2c4abc None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:25.065804 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ccd202-4630-40f3-b2b2-c1669b2c4abc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:25.072417 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ccd202-4630-40f3-b2b2-c1669b2c4abc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:25.074753 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ccd202-4630-40f3-b2b2-c1669b2c4abc None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:25.088976 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-21ccd202-4630-40f3-b2b2-c1669b2c4abc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:32:25.088976 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:25.088976 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:25.088976 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:25.088976 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:25.088976 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:25.088976 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:25.088976 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:25.088976 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:25.088976 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:25.088976 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:25.088976 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:25.088976 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:32:25.088976 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:32:25.088976 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:32:25.090080 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:32:25.090080 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:25.090080 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:25.090080 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:25.090080 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:25.090080 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:25.090080 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:25.090080 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:25.090080 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:25.090080 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:25.090080 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:25.090080 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:25.090080 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:25.090080 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:25.090080 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:25.090080 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:25.090080 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:25.095691 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:32:25.095691 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:32:25.095691 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:32:25.095691 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:32:25.095691 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:32:25.095691 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:32:25.095691 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:32:25.095691 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:32:25.095691 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 490/987] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jun 15 14:32:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:25.096217 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac254267-6aa9-44f9-aa98-2c7a16ca596f None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:25.128956 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:25.200410 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac254267-6aa9-44f9-aa98-2c7a16ca596f None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:25.235200 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f28ff2-1cc4-4ed7-9982-cd150261a254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:25.235860 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f28ff2-1cc4-4ed7-9982-cd150261a254 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:25.236182 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f28ff2-1cc4-4ed7-9982-cd150261a254 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:25.236465 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f28ff2-1cc4-4ed7-9982-cd150261a254 None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:25.237250 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f28ff2-1cc4-4ed7-9982-cd150261a254 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:25.237562 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f28ff2-1cc4-4ed7-9982-cd150261a254 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:25.238984 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f28ff2-1cc4-4ed7-9982-cd150261a254 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:25.256172 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 498/988] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jun 15 14:32:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:25.257838 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ccd202-4630-40f3-b2b2-c1669b2c4abc None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:25.361023 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ccd202-4630-40f3-b2b2-c1669b2c4abc None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:25.362552 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:25.397295 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a032249-2d6d-4df7-a70c-520b4f53b92b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:25.397860 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a032249-2d6d-4df7-a70c-520b4f53b92b None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:25.398137 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a032249-2d6d-4df7-a70c-520b4f53b92b None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:25.398424 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a032249-2d6d-4df7-a70c-520b4f53b92b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:25.399585 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a032249-2d6d-4df7-a70c-520b4f53b92b None admin] RBAC: Adding query filter params (role.id=76f33d89f76347be9ef2fdac0435173f, scope.project.id=3094ff8966f149bda784da6a51b1358f, user.id=19d82c04f9994654adfb10f4bd0573c2) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:25.400032 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a032249-2d6d-4df7-a70c-520b4f53b92b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:25.402346 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a032249-2d6d-4df7-a70c-520b4f53b92b None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:25.416127 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 491/989] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jun 15 14:32:25 2022] GET /identity/v3/role_assignments?user.id=19d82c04f9994654adfb10f4bd0573c2&scope.project.id=3094ff8966f149bda784da6a51b1358f&role.id=76f33d89f76347be9ef2fdac0435173f => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:25.677679 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d61d5-277f-4e85-a48c-3f85d14869de None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:25.678045 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d61d5-277f-4e85-a48c-3f85d14869de None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:25.678419 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d61d5-277f-4e85-a48c-3f85d14869de None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:25.679961 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 499/990] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:32:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:32:25.685530 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75587c7-d4a2-4cef-a12a-ee71d90ed370 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:25.685820 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75587c7-d4a2-4cef-a12a-ee71d90ed370 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:25.686099 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75587c7-d4a2-4cef-a12a-ee71d90ed370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:25.703611 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-a75587c7-d4a2-4cef-a12a-ee71d90ed370 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:32:25.704955 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-a75587c7-d4a2-4cef-a12a-ee71d90ed370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:32:25.780159 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 492/991] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:32:25 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:32:25.799896 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9d61d5-277f-4e85-a48c-3f85d14869de None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:25.898895 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:25.909971 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9d61d5-277f-4e85-a48c-3f85d14869de None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:25.946939 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6479a47c-c5db-4d73-bee1-483cc53b255c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:25.948311 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6479a47c-c5db-4d73-bee1-483cc53b255c None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:25.948656 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6479a47c-c5db-4d73-bee1-483cc53b255c None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:25.948988 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6479a47c-c5db-4d73-bee1-483cc53b255c None admin] PATH_INFO: `/v3/services/compute` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:25.949698 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6479a47c-c5db-4d73-bee1-483cc53b255c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:25.951358 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6479a47c-c5db-4d73-bee1-483cc53b255c None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:25.964864 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-6479a47c-c5db-4d73-bee1-483cc53b255c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 14:32:25.964864 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:25.964864 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:32:25.964864 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:32:25.964864 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:32:25.964864 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:32:25.964864 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:32:25.964864 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:32:25.964864 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 14:32:25.964864 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:25.964864 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:32:25.964864 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:25.964864 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:25.964864 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:25.964864 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:25.964864 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:25.964864 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:25.966044 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:25.966044 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:25.966044 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:25.966044 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:25.966044 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:25.966044 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:25.966044 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:32:25.966044 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:32:25.966044 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:32:25.966044 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:32:25.966044 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:25.966044 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:25.966044 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:25.966044 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:25.966044 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:25.966044 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:25.967103 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:25.967103 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:25.967103 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:25.967103 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:25.967103 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:25.967103 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:25.967103 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:25.967103 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:25.967103 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:25.967103 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:25.967103 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:32:25.967103 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:32:25.967103 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 14:32:25.967103 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:25.969290 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 500/992] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jun 15 14:32:25 2022] GET /identity/v3/services/compute => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:32:25.975240 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75587c7-d4a2-4cef-a12a-ee71d90ed370 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:26.081264 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:26.088039 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75587c7-d4a2-4cef-a12a-ee71d90ed370 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:26.127334 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6355e197-7ec1-408d-bc5c-f0945a834305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:26.128001 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6355e197-7ec1-408d-bc5c-f0945a834305 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:26.128348 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6355e197-7ec1-408d-bc5c-f0945a834305 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:26.128788 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6355e197-7ec1-408d-bc5c-f0945a834305 None admin] PATH_INFO: `/v3/services` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:26.129493 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6355e197-7ec1-408d-bc5c-f0945a834305 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:26.129797 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6355e197-7ec1-408d-bc5c-f0945a834305 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:26.131155 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6355e197-7ec1-408d-bc5c-f0945a834305 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:26.136772 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 493/993] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Jun 15 14:32:25 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:26.142294 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6479a47c-c5db-4d73-bee1-483cc53b255c None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:26.253103 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6479a47c-c5db-4d73-bee1-483cc53b255c None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:26.289404 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc63cd3d-0d58-44cc-a252-30431fd354e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:26.290231 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc63cd3d-0d58-44cc-a252-30431fd354e7 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:26.290516 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc63cd3d-0d58-44cc-a252-30431fd354e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:26.290793 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc63cd3d-0d58-44cc-a252-30431fd354e7 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:26.291567 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc63cd3d-0d58-44cc-a252-30431fd354e7 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:26.291880 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc63cd3d-0d58-44cc-a252-30431fd354e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:26.293296 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc63cd3d-0d58-44cc-a252-30431fd354e7 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:26.303614 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 501/994] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Jun 15 14:32:26 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:26.304591 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6355e197-7ec1-408d-bc5c-f0945a834305 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:26.408877 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6355e197-7ec1-408d-bc5c-f0945a834305 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:26.445017 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b98b027-9357-4d3d-8a6d-7c37e1572512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:26.445844 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b98b027-9357-4d3d-8a6d-7c37e1572512 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:26.446126 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b98b027-9357-4d3d-8a6d-7c37e1572512 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:26.446399 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b98b027-9357-4d3d-8a6d-7c37e1572512 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:26.447183 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b98b027-9357-4d3d-8a6d-7c37e1572512 None admin] RBAC: Adding query filter params (interface=public, service_id=71d52af181ac48e48e35f39d15eea4c4, region_id=RegionOne) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:26.447495 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b98b027-9357-4d3d-8a6d-7c37e1572512 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:26.449449 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b98b027-9357-4d3d-8a6d-7c37e1572512 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:26.459345 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 494/995] 10.222.0.42 () {56 vars in 1251 bytes} [Wed Jun 15 14:32:26 2022] GET /identity/v3/endpoints?service_id=71d52af181ac48e48e35f39d15eea4c4&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:32:26.460477 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc63cd3d-0d58-44cc-a252-30431fd354e7 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:26.491720 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:26.581530 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc63cd3d-0d58-44cc-a252-30431fd354e7 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:26.616471 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa4c454-54cb-484a-8c4f-abb2a833b291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:26.617135 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa4c454-54cb-484a-8c4f-abb2a833b291 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:26.617417 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa4c454-54cb-484a-8c4f-abb2a833b291 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:26.617688 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa4c454-54cb-484a-8c4f-abb2a833b291 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:26.618352 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa4c454-54cb-484a-8c4f-abb2a833b291 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:26.619703 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa4c454-54cb-484a-8c4f-abb2a833b291 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:26.632159 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 502/996] 10.222.0.42 () {56 vars in 1087 bytes} [Wed Jun 15 14:32:26 2022] GET /identity/v3/services => generated 1727 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:32:26.632377 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae46770-d181-4a4f-9431-6690589b40d9 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:26.632377 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae46770-d181-4a4f-9431-6690589b40d9 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:26.632546 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae46770-d181-4a4f-9431-6690589b40d9 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:26.633770 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 495/997] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:32:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:32:26.640496 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8c6c27-e02a-4dca-81ba-87180a8c541a None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:26.640837 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8c6c27-e02a-4dca-81ba-87180a8c541a None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:26.641057 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8c6c27-e02a-4dca-81ba-87180a8c541a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:26.659448 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-0e8c6c27-e02a-4dca-81ba-87180a8c541a None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:32:26.660713 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-0e8c6c27-e02a-4dca-81ba-87180a8c541a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:32:26.739214 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 503/998] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:32:26 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:32:26.763998 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae46770-d181-4a4f-9431-6690589b40d9 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:26.793186 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:26.876130 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae46770-d181-4a4f-9431-6690589b40d9 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:26.910530 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8533fca5-fd1f-44ed-bd2f-b19b62a803ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:26.911376 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8533fca5-fd1f-44ed-bd2f-b19b62a803ac None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:26.911662 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8533fca5-fd1f-44ed-bd2f-b19b62a803ac None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:26.911936 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8533fca5-fd1f-44ed-bd2f-b19b62a803ac None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:26.912609 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8533fca5-fd1f-44ed-bd2f-b19b62a803ac None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:26.914023 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8533fca5-fd1f-44ed-bd2f-b19b62a803ac None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:26.926771 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-8533fca5-fd1f-44ed-bd2f-b19b62a803ac None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 14:32:26.926771 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:26.926771 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:32:26.926771 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:32:26.926771 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:32:26.926771 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:32:26.926771 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:32:26.926771 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:32:26.926771 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 14:32:26.926771 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:32:26.926771 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:32:26.926771 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:32:26.926771 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:26.926771 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:26.926771 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:26.926771 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:26.926771 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:26.927779 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:26.927779 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:26.927779 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:26.927779 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:26.927779 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:26.927779 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:26.927779 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:32:26.927779 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:32:26.927779 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:32:26.927779 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:32:26.927779 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:26.927779 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:26.927779 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:26.927779 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:26.927779 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:26.927779 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:26.929087 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:26.929087 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:26.929087 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:26.929087 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:26.929087 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:26.929087 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:26.929087 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:26.929087 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:26.929087 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:26.929087 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:26.929087 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:32:26.929087 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:32:26.929087 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 14:32:26.929087 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:32:26.929087 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 496/999] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Jun 15 14:32:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:26.935001 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8c6c27-e02a-4dca-81ba-87180a8c541a None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:26.950480 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:27.035560 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8c6c27-e02a-4dca-81ba-87180a8c541a None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:27.070496 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7a90a8-6189-472a-be11-639453c86c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:27.071417 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7a90a8-6189-472a-be11-639453c86c6f None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:27.071875 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7a90a8-6189-472a-be11-639453c86c6f None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:27.072347 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7a90a8-6189-472a-be11-639453c86c6f None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:27.073245 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7a90a8-6189-472a-be11-639453c86c6f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:27.073768 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7a90a8-6189-472a-be11-639453c86c6f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:27.075291 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7a90a8-6189-472a-be11-639453c86c6f None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:27.080447 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 504/1000] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Jun 15 14:32:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:27.086290 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8533fca5-fd1f-44ed-bd2f-b19b62a803ac None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:27.176902 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:27.189470 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8533fca5-fd1f-44ed-bd2f-b19b62a803ac None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:27.222776 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03183712-82f4-4543-95f0-aeaf3d15516f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:27.223435 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03183712-82f4-4543-95f0-aeaf3d15516f None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:27.223712 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03183712-82f4-4543-95f0-aeaf3d15516f None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:27.223981 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03183712-82f4-4543-95f0-aeaf3d15516f None admin] PATH_INFO: `/v3/services` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:27.224713 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03183712-82f4-4543-95f0-aeaf3d15516f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:27.225014 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03183712-82f4-4543-95f0-aeaf3d15516f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:27.226361 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03183712-82f4-4543-95f0-aeaf3d15516f None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:27.232324 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 497/1001] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Jun 15 14:32:27 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:27.237679 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7a90a8-6189-472a-be11-639453c86c6f None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:27.317530 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:27.340588 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7a90a8-6189-472a-be11-639453c86c6f None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:27.376473 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b3c9d1-3cdc-4957-97e8-4339059cfa93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:27.377375 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b3c9d1-3cdc-4957-97e8-4339059cfa93 None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:27.377836 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b3c9d1-3cdc-4957-97e8-4339059cfa93 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:27.378270 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b3c9d1-3cdc-4957-97e8-4339059cfa93 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:27.379161 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b3c9d1-3cdc-4957-97e8-4339059cfa93 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:27.380691 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b3c9d1-3cdc-4957-97e8-4339059cfa93 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:27.462645 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 505/1002] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Jun 15 14:32:27 2022] POST /identity/v3/endpoints => generated 325 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:32:27.792976 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4341fd-c24e-4503-958a-ac1066740e02 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:27.793379 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4341fd-c24e-4503-958a-ac1066740e02 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:27.793555 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4341fd-c24e-4503-958a-ac1066740e02 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:27.795081 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 498/1003] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:32:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:32:27.800130 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d2d3ad-d690-47c5-9868-3b04522b0495 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:27.800436 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d2d3ad-d690-47c5-9868-3b04522b0495 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:27.800719 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d2d3ad-d690-47c5-9868-3b04522b0495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:27.817679 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-34d2d3ad-d690-47c5-9868-3b04522b0495 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:32:27.818876 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-34d2d3ad-d690-47c5-9868-3b04522b0495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:32:27.896165 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 506/1004] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:32:27 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:32:27.919461 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4341fd-c24e-4503-958a-ac1066740e02 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:28.027824 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4341fd-c24e-4503-958a-ac1066740e02 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:28.061224 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5e62e3-82fc-451e-b2a6-d7cc8438f7c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:28.061944 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e62e3-82fc-451e-b2a6-d7cc8438f7c8 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:28.062227 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e62e3-82fc-451e-b2a6-d7cc8438f7c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:28.062502 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e62e3-82fc-451e-b2a6-d7cc8438f7c8 None admin] PATH_INFO: `/v3/services/image` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:28.063144 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5e62e3-82fc-451e-b2a6-d7cc8438f7c8 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:28.064543 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5e62e3-82fc-451e-b2a6-d7cc8438f7c8 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:28.076616 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-ff5e62e3-82fc-451e-b2a6-d7cc8438f7c8 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 14:32:28.076616 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:28.076616 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:32:28.076616 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:32:28.076616 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:32:28.076616 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:32:28.076616 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:32:28.076616 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:32:28.076616 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 14:32:28.076616 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:32:28.076616 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:32:28.076616 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:32:28.076616 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:28.076616 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:28.076616 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:28.076616 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:28.076616 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:28.077614 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:28.077614 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:28.077614 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:28.077614 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:28.077614 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:28.077614 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:28.077614 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:32:28.077614 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:32:28.077614 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:32:28.077614 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:32:28.077614 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:28.077614 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:28.077614 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:28.077614 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:28.077614 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:28.077614 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:28.078431 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:28.078431 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:28.078431 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:28.078431 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:28.078431 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:28.078431 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:28.078431 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:28.078431 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:28.078431 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:28.078431 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:28.078431 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:32:28.078431 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:32:28.078431 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 14:32:28.078431 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:32:28.079937 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 499/1005] 10.222.0.42 () {56 vars in 1105 bytes} [Wed Jun 15 14:32:27 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:28.083649 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d2d3ad-d690-47c5-9868-3b04522b0495 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:28.145681 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:28.186017 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d2d3ad-d690-47c5-9868-3b04522b0495 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:28.221516 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29064daa-1167-4a29-9d77-fb6148b9bf39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:28.222185 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29064daa-1167-4a29-9d77-fb6148b9bf39 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:28.222470 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29064daa-1167-4a29-9d77-fb6148b9bf39 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:28.222751 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29064daa-1167-4a29-9d77-fb6148b9bf39 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:28.223474 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29064daa-1167-4a29-9d77-fb6148b9bf39 None admin] RBAC: Adding query filter params (name=image) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:28.223773 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29064daa-1167-4a29-9d77-fb6148b9bf39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:28.225490 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29064daa-1167-4a29-9d77-fb6148b9bf39 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:28.235455 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 507/1006] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:32:28 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:28.236984 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5e62e3-82fc-451e-b2a6-d7cc8438f7c8 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:28.257409 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:28.342165 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5e62e3-82fc-451e-b2a6-d7cc8438f7c8 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:28.376371 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40522153-daea-4b1e-acf3-b560dbab7a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:28.377029 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40522153-daea-4b1e-acf3-b560dbab7a61 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:28.377307 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40522153-daea-4b1e-acf3-b560dbab7a61 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:28.377577 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40522153-daea-4b1e-acf3-b560dbab7a61 None admin] PATH_INFO: `/v3/services` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:28.378271 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40522153-daea-4b1e-acf3-b560dbab7a61 None admin] RBAC: Adding query filter params (type=image) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:28.378576 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40522153-daea-4b1e-acf3-b560dbab7a61 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:28.379923 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40522153-daea-4b1e-acf3-b560dbab7a61 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:28.384641 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 500/1007] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:32:28 2022] GET /identity/v3/services?type=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:28.791177 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce71715-e780-46d2-8fb2-d9ac88443b3d None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:28.791719 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce71715-e780-46d2-8fb2-d9ac88443b3d None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:28.791719 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce71715-e780-46d2-8fb2-d9ac88443b3d None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:28.793633 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 508/1008] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:32:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:32:28.798931 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c040418b-4d7d-4ba8-b112-ea57f64a7d6a None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:28.799223 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c040418b-4d7d-4ba8-b112-ea57f64a7d6a None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:28.799502 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c040418b-4d7d-4ba8-b112-ea57f64a7d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:28.816957 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-c040418b-4d7d-4ba8-b112-ea57f64a7d6a None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:32:28.818143 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-c040418b-4d7d-4ba8-b112-ea57f64a7d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:32:28.890483 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 501/1009] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:32:28 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:32:28.913855 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce71715-e780-46d2-8fb2-d9ac88443b3d None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:29.020607 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce71715-e780-46d2-8fb2-d9ac88443b3d None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:29.048408 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:29.056639 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede581f3-0243-4d88-a574-2865102c857d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:29.057441 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede581f3-0243-4d88-a574-2865102c857d None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:29.058351 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede581f3-0243-4d88-a574-2865102c857d None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:29.058859 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede581f3-0243-4d88-a574-2865102c857d None admin] PATH_INFO: `/v3/services/compute` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:29.059620 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede581f3-0243-4d88-a574-2865102c857d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:29.061008 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede581f3-0243-4d88-a574-2865102c857d None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:29.073047 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-ede581f3-0243-4d88-a574-2865102c857d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 14:32:29.073047 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:29.073047 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:32:29.073047 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:32:29.073047 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:32:29.073047 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:32:29.073047 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:32:29.073047 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:32:29.073047 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 14:32:29.073047 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:29.073047 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:32:29.073047 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:29.073047 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:29.073047 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:29.073047 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:29.073047 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:29.073047 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:29.074118 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:29.074118 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:29.074118 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:29.074118 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:29.074118 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:29.074118 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:29.074118 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:32:29.074118 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:32:29.074118 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:32:29.074118 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:32:29.074118 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:29.074118 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:29.074118 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:29.074118 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:29.074118 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:29.074118 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:29.074989 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:29.074989 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:29.074989 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:29.074989 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:29.074989 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:29.074989 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:29.074989 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:29.074989 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:29.074989 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:29.074989 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:29.074989 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:32:29.074989 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:32:29.074989 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 14:32:29.074989 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:29.079713 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 509/1010] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jun 15 14:32:28 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:29.080740 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c040418b-4d7d-4ba8-b112-ea57f64a7d6a None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:29.093053 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:29.184305 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c040418b-4d7d-4ba8-b112-ea57f64a7d6a None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:29.219543 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78bd875-5d15-4ce8-8137-21146d87d67c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:29.220235 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78bd875-5d15-4ce8-8137-21146d87d67c None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:29.220525 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78bd875-5d15-4ce8-8137-21146d87d67c None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:29.220806 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78bd875-5d15-4ce8-8137-21146d87d67c None admin] PATH_INFO: `/v3/services` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:29.221510 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78bd875-5d15-4ce8-8137-21146d87d67c None admin] RBAC: Adding query filter params (name=compute) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:29.221820 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78bd875-5d15-4ce8-8137-21146d87d67c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:29.223192 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78bd875-5d15-4ce8-8137-21146d87d67c None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:29.233094 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 502/1011] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Jun 15 14:32:29 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:29.234459 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede581f3-0243-4d88-a574-2865102c857d None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:29.335569 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede581f3-0243-4d88-a574-2865102c857d None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:29.369589 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51924f65-d58c-401b-bb04-3d0a2eea8cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:29.370906 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51924f65-d58c-401b-bb04-3d0a2eea8cc7 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:29.371418 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51924f65-d58c-401b-bb04-3d0a2eea8cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:29.371709 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51924f65-d58c-401b-bb04-3d0a2eea8cc7 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:29.372455 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51924f65-d58c-401b-bb04-3d0a2eea8cc7 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:29.372769 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51924f65-d58c-401b-bb04-3d0a2eea8cc7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:29.374123 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51924f65-d58c-401b-bb04-3d0a2eea8cc7 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:29.384961 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 510/1012] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Jun 15 14:32:29 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:32:29.385121 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78bd875-5d15-4ce8-8137-21146d87d67c None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:29.462362 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:29.491154 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78bd875-5d15-4ce8-8137-21146d87d67c None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:29.528390 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e41aad8-82bf-407f-bbca-5b6c70e4c01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:29.529046 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e41aad8-82bf-407f-bbca-5b6c70e4c01a None admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:29.529326 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e41aad8-82bf-407f-bbca-5b6c70e4c01a None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:29.529597 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e41aad8-82bf-407f-bbca-5b6c70e4c01a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:29.530327 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e41aad8-82bf-407f-bbca-5b6c70e4c01a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:29.531675 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e41aad8-82bf-407f-bbca-5b6c70e4c01a None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:29.605623 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 503/1013] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Jun 15 14:32:29 2022] POST /identity/v3/endpoints => generated 334 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:32:29.633050 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7a5de-75a5-4cbc-a699-89c5d8fa1e74 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:29.633482 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7a5de-75a5-4cbc-a699-89c5d8fa1e74 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:29.633906 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7a5de-75a5-4cbc-a699-89c5d8fa1e74 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:29.635520 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 511/1014] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:32:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:32:29.642267 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c2365-0b8c-42ff-b724-bee3affa5296 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:29.642552 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c2365-0b8c-42ff-b724-bee3affa5296 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:29.642891 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c2365-0b8c-42ff-b724-bee3affa5296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:29.661093 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-5e7c2365-0b8c-42ff-b724-bee3affa5296 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:32:29.662311 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-5e7c2365-0b8c-42ff-b724-bee3affa5296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:32:29.754312 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 504/1015] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:32:29 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:32:29.778154 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a7a5de-75a5-4cbc-a699-89c5d8fa1e74 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:29.839752 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:29.886956 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a7a5de-75a5-4cbc-a699-89c5d8fa1e74 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:29.924213 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f57f138-c4fc-46e9-adb3-ae548cf6d5ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:29.925013 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57f138-c4fc-46e9-adb3-ae548cf6d5ad None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:29.925291 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57f138-c4fc-46e9-adb3-ae548cf6d5ad None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:29.925558 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57f138-c4fc-46e9-adb3-ae548cf6d5ad None admin] PATH_INFO: `/v3/domains/default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:29.926716 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f57f138-c4fc-46e9-adb3-ae548cf6d5ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:29.928157 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f57f138-c4fc-46e9-adb3-ae548cf6d5ad None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:29.929933 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 512/1016] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:32:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:29.936472 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7c2365-0b8c-42ff-b724-bee3affa5296 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:29.998069 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:30.041688 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7c2365-0b8c-42ff-b724-bee3affa5296 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:30.078568 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f858907-5be9-4f6a-be06-7ae8c05c8ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:30.079239 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f858907-5be9-4f6a-be06-7ae8c05c8ad7 None admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:30.079523 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f858907-5be9-4f6a-be06-7ae8c05c8ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:30.079798 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f858907-5be9-4f6a-be06-7ae8c05c8ad7 None admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:30.080533 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f858907-5be9-4f6a-be06-7ae8c05c8ad7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:30.081960 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f858907-5be9-4f6a-be06-7ae8c05c8ad7 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:30.183409 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 505/1017] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Jun 15 14:32:29 2022] POST /identity/v3/projects => generated 311 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:32:30.573266 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48141646-8273-426a-8275-c1d2015d186d None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:30.573676 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48141646-8273-426a-8275-c1d2015d186d None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:30.574195 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48141646-8273-426a-8275-c1d2015d186d None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:30.575649 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 513/1018] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:32:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:32:30.580989 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9922c9e-db50-414b-8b16-f0a67a297161 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:30.581449 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9922c9e-db50-414b-8b16-f0a67a297161 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:30.581628 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9922c9e-db50-414b-8b16-f0a67a297161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:30.598376 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-c9922c9e-db50-414b-8b16-f0a67a297161 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:32:30.599537 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-c9922c9e-db50-414b-8b16-f0a67a297161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:32:30.668130 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 506/1019] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:32:30 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:32:30.690112 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48141646-8273-426a-8275-c1d2015d186d None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:30.793483 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48141646-8273-426a-8275-c1d2015d186d None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:30.818587 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:30.830686 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75721467-c060-4649-85a4-1d17fe7b6206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:30.831354 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75721467-c060-4649-85a4-1d17fe7b6206 None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:30.831639 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75721467-c060-4649-85a4-1d17fe7b6206 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:30.831918 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75721467-c060-4649-85a4-1d17fe7b6206 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:30.839354 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75721467-c060-4649-85a4-1d17fe7b6206 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:30.839354 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75721467-c060-4649-85a4-1d17fe7b6206 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:30.931229 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 514/1020] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Jun 15 14:32:30 2022] POST /identity/v3/services => generated 242 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:32:31.862207 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4418dff-64bc-4c2b-950e-c4c0de393006 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:31.862757 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4418dff-64bc-4c2b-950e-c4c0de393006 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:31.862757 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4418dff-64bc-4c2b-950e-c4c0de393006 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:31.864202 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 507/1021] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:32:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:32:31.869102 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b02507e-bda1-4141-a2d9-87f30d5c129b None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:31.869393 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b02507e-bda1-4141-a2d9-87f30d5c129b None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:31.869669 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b02507e-bda1-4141-a2d9-87f30d5c129b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:31.887474 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-2b02507e-bda1-4141-a2d9-87f30d5c129b None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:32:31.888660 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-2b02507e-bda1-4141-a2d9-87f30d5c129b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:32:31.965893 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 515/1022] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:32:31 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:32:31.988010 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4418dff-64bc-4c2b-950e-c4c0de393006 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:32.051451 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:32.112664 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4418dff-64bc-4c2b-950e-c4c0de393006 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:32.147003 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1f6492-5d1b-4c04-9a48-537be38e6a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:32.147873 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f6492-5d1b-4c04-9a48-537be38e6a86 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:32.148185 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f6492-5d1b-4c04-9a48-537be38e6a86 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:32.148464 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f6492-5d1b-4c04-9a48-537be38e6a86 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:32.159167 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1f6492-5d1b-4c04-9a48-537be38e6a86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:32:32.167869 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-4d1f6492-5d1b-4c04-9a48-537be38e6a86 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 14:32:32.167869 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:32.167869 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:32.167869 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:32.167869 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:32.167869 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:32.167869 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:32.167869 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:32.167869 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:32.167869 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:32.167869 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:32.167869 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:32.167869 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:32:32.167869 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:32:32.167869 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:32:32.167869 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:32:32.168829 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:32:32.168829 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:32:32.168829 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:32:32.168829 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:32:32.168829 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:32.168829 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:32.168829 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:32.168829 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:32.168829 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:32.168829 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:32.168829 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:32.168829 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:32.168829 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:32.168829 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:32.168829 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:32.168829 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:32.169834 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:32.169834 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:32.169834 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:32.169834 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:32.169834 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:32:32.169834 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:32:32.169834 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:32:32.169834 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:32:32.169834 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:32:32.169834 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:32:32.169834 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 14:32:32.169834 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:32:32.169834 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 508/1023] 10.222.0.42 () {56 vars in 1120 bytes} [Wed Jun 15 14:32:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:32.175089 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b02507e-bda1-4141-a2d9-87f30d5c129b None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:32.275129 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b02507e-bda1-4141-a2d9-87f30d5c129b None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:32.285896 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:32.312923 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd605cca-3585-4a4f-af88-2c11f3bb693a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:32.313804 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd605cca-3585-4a4f-af88-2c11f3bb693a None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:32.314070 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd605cca-3585-4a4f-af88-2c11f3bb693a None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:32.314386 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd605cca-3585-4a4f-af88-2c11f3bb693a None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:32.315160 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd605cca-3585-4a4f-af88-2c11f3bb693a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:32.315500 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd605cca-3585-4a4f-af88-2c11f3bb693a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:32.316932 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd605cca-3585-4a4f-af88-2c11f3bb693a None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:32.331714 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 516/1024] 10.222.0.42 () {56 vars in 1115 bytes} [Wed Jun 15 14:32:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:32.332926 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1f6492-5d1b-4c04-9a48-537be38e6a86 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:32.365436 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efbd03b-579a-46b5-b8dd-8ffcf51cadf0 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:32.365721 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efbd03b-579a-46b5-b8dd-8ffcf51cadf0 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:32.365918 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efbd03b-579a-46b5-b8dd-8ffcf51cadf0 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:32.367290 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 517/1025] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:32:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:32:32.373364 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c383d8d7-7e68-46c1-bae6-eaef66c6d794 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:32.373659 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c383d8d7-7e68-46c1-bae6-eaef66c6d794 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:32.373944 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c383d8d7-7e68-46c1-bae6-eaef66c6d794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:32.392786 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-c383d8d7-7e68-46c1-bae6-eaef66c6d794 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:32:32.393977 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c383d8d7-7e68-46c1-bae6-eaef66c6d794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:32:32.437052 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:32.441246 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1f6492-5d1b-4c04-9a48-537be38e6a86 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:32.468137 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 518/1026] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:32:32 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:32:32.476655 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c8dd4e-e726-4d16-b948-108300a59b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:32.477456 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c8dd4e-e726-4d16-b948-108300a59b78 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:32.477738 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c8dd4e-e726-4d16-b948-108300a59b78 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:32.478015 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c8dd4e-e726-4d16-b948-108300a59b78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:32.489544 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c8dd4e-e726-4d16-b948-108300a59b78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:32.490987 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c8dd4e-e726-4d16-b948-108300a59b78 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:32.493016 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c383d8d7-7e68-46c1-bae6-eaef66c6d794 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:32.508905 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-34c8dd4e-e726-4d16-b948-108300a59b78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:32.508905 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:32.508905 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:32.508905 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:32.508905 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:32.508905 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:32.508905 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:32.508905 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:32.508905 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:32.508905 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:32.508905 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:32.508905 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:32.508905 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:32:32.508905 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:32:32.508905 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:32:32.509844 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:32:32.509844 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:32.509844 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:32.509844 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:32.509844 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:32.509844 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:32.509844 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:32.509844 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:32.509844 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:32.509844 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:32.509844 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:32.509844 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:32.509844 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:32.509844 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:32.509844 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:32.509844 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:32.509844 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:32.516435 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:32:32.516435 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:32:32.516435 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:32.516435 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:32:32.516435 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 509/1027] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:32:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:32.516435 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c8dd4e-e726-4d16-b948-108300a59b78 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:32.608002 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c383d8d7-7e68-46c1-bae6-eaef66c6d794 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:32.609810 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:32.627572 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c8dd4e-e726-4d16-b948-108300a59b78 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:32.646868 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162132e5-7f6d-4cc3-aa82-c559c13efb93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:32.647671 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162132e5-7f6d-4cc3-aa82-c559c13efb93 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:32.647948 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162132e5-7f6d-4cc3-aa82-c559c13efb93 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:32.648266 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162132e5-7f6d-4cc3-aa82-c559c13efb93 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:32.650114 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162132e5-7f6d-4cc3-aa82-c559c13efb93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:32.651727 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162132e5-7f6d-4cc3-aa82-c559c13efb93 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:32.657917 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 519/1028] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:32:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:32:32.659336 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162132e5-7f6d-4cc3-aa82-c559c13efb93 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:32.663779 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d496ed6c-6cf3-4c3c-a995-38bfbbf1fdb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:32.664561 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d496ed6c-6cf3-4c3c-a995-38bfbbf1fdb0 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:32.664843 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d496ed6c-6cf3-4c3c-a995-38bfbbf1fdb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:32.665114 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d496ed6c-6cf3-4c3c-a995-38bfbbf1fdb0 None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:32.665830 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d496ed6c-6cf3-4c3c-a995-38bfbbf1fdb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:32.666137 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d496ed6c-6cf3-4c3c-a995-38bfbbf1fdb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:32.671688 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d496ed6c-6cf3-4c3c-a995-38bfbbf1fdb0 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:32.689023 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 510/1029] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:32:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:32:32.690004 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d496ed6c-6cf3-4c3c-a995-38bfbbf1fdb0 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:32.703864 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:32.772858 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162132e5-7f6d-4cc3-aa82-c559c13efb93 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:32.798584 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d496ed6c-6cf3-4c3c-a995-38bfbbf1fdb0 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:32.812376 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb33611b-0c57-42f9-bee1-d8b2a78373d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:32.813248 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb33611b-0c57-42f9-bee1-d8b2a78373d4 None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:32.813709 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb33611b-0c57-42f9-bee1-d8b2a78373d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:32.814050 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb33611b-0c57-42f9-bee1-d8b2a78373d4 None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:32.814830 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb33611b-0c57-42f9-bee1-d8b2a78373d4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:32.816453 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb33611b-0c57-42f9-bee1-d8b2a78373d4 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:32.834368 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c32750-c871-44fc-8d51-30290f7658e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:32.835154 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c32750-c871-44fc-8d51-30290f7658e5 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:32.835432 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c32750-c871-44fc-8d51-30290f7658e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:32.835704 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c32750-c871-44fc-8d51-30290f7658e5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:32.843345 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c32750-c871-44fc-8d51-30290f7658e5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:32.844798 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c32750-c871-44fc-8d51-30290f7658e5 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:32.859025 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-b5c32750-c871-44fc-8d51-30290f7658e5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:32:32.859025 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:32.859025 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:32.859025 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:32.859025 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:32.859025 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:32.859025 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:32.859025 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:32.859025 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:32.859025 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:32.859025 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:32.859025 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:32.859025 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:32:32.859025 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:32:32.859025 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:32:32.859025 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:32:32.860112 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:32.860112 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:32.860112 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:32:32.860112 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:32:32.860112 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:32:32.860112 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:32:32.860112 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:32.860112 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:32.860112 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:32.860112 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:32.860112 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:32.860112 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:32.860112 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:32.860112 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:32.860112 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:32.860112 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:32.860112 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:32.860112 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:32.861093 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:32.861093 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:32.861093 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:32:32.861093 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:32:32.861093 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:32:32.861093 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:32:32.861093 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:32:32.861093 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:32:32.861093 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:32:32.861093 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:32:32.861093 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 511/1030] 10.222.0.42 () {56 vars in 1093 bytes} [Wed Jun 15 14:32:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:32.865938 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c32750-c871-44fc-8d51-30290f7658e5 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:32.931544 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 520/1031] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Jun 15 14:32:32 2022] POST /identity/v3/users => generated 285 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:32:32.940158 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:32.969049 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c32750-c871-44fc-8d51-30290f7658e5 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:33.008320 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dac614-3046-40d6-8df0-999d4d761069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:33.009103 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dac614-3046-40d6-8df0-999d4d761069 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:33.009452 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dac614-3046-40d6-8df0-999d4d761069 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:33.009796 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dac614-3046-40d6-8df0-999d4d761069 None admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:33.011256 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dac614-3046-40d6-8df0-999d4d761069 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:33.011654 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dac614-3046-40d6-8df0-999d4d761069 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:33.013263 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dac614-3046-40d6-8df0-999d4d761069 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:33.039229 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 512/1032] 10.222.0.42 () {56 vars in 1133 bytes} [Wed Jun 15 14:32:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:32:33.040606 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb33611b-0c57-42f9-bee1-d8b2a78373d4 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:33.073871 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:33.099184 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e301f67f-f1f6-465d-9acc-dcbb36aa8afc None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:33.099474 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e301f67f-f1f6-465d-9acc-dcbb36aa8afc None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:33.099752 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e301f67f-f1f6-465d-9acc-dcbb36aa8afc None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:33.101162 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 513/1033] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:32:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:32:33.107415 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e58af1-5311-4623-8278-5a3d1070188f None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:33.107723 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e58af1-5311-4623-8278-5a3d1070188f None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:33.107981 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e58af1-5311-4623-8278-5a3d1070188f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:33.129251 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-42e58af1-5311-4623-8278-5a3d1070188f None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:32:33.130452 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-42e58af1-5311-4623-8278-5a3d1070188f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:32:33.161683 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb33611b-0c57-42f9-bee1-d8b2a78373d4 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:33.205968 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a297e93-bca6-4fb3-9e56-bb52b131b656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:33.206739 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a297e93-bca6-4fb3-9e56-bb52b131b656 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:33.207149 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a297e93-bca6-4fb3-9e56-bb52b131b656 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:33.207526 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a297e93-bca6-4fb3-9e56-bb52b131b656 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:33.208328 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 514/1034] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:32:33 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:32:33.232104 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e58af1-5311-4623-8278-5a3d1070188f None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:33.284964 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a297e93-bca6-4fb3-9e56-bb52b131b656 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:33.286416 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a297e93-bca6-4fb3-9e56-bb52b131b656 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:33.297115 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:33.313377 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:33.341149 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e58af1-5311-4623-8278-5a3d1070188f None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:33.378989 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9754dc4-edf7-40c9-81db-a6d7bddc428b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:33.379775 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9754dc4-edf7-40c9-81db-a6d7bddc428b None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:33.380057 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9754dc4-edf7-40c9-81db-a6d7bddc428b None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:33.380364 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9754dc4-edf7-40c9-81db-a6d7bddc428b None admin] PATH_INFO: `/v3/services/image` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:33.381123 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9754dc4-edf7-40c9-81db-a6d7bddc428b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:33.382601 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9754dc4-edf7-40c9-81db-a6d7bddc428b None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:33.392881 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 521/1035] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jun 15 14:32:33 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:32:33.393903 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a297e93-bca6-4fb3-9e56-bb52b131b656 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:33.395291 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-e9754dc4-edf7-40c9-81db-a6d7bddc428b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 14:32:33.395291 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:33.395291 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:32:33.395291 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:32:33.395291 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:32:33.395291 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:32:33.395291 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:32:33.395291 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:32:33.395291 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 14:32:33.395291 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:32:33.395291 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:32:33.395291 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:32:33.395291 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:33.395291 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:33.395291 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:33.395291 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:33.395291 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:33.396340 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:33.396340 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:33.396340 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:33.396340 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:33.396340 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:33.396340 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:33.396340 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:32:33.396340 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:32:33.396340 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:32:33.396340 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:32:33.396340 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:33.396340 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:33.396340 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:33.396340 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:33.396340 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:33.396340 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:33.397193 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:33.397193 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:33.397193 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:33.397193 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:33.397193 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:33.397193 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:33.397193 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:33.397193 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:33.397193 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:33.397193 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:33.397193 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:32:33.397193 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:32:33.397193 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 14:32:33.397193 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:32:33.397193 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 515/1036] 10.222.0.42 () {56 vars in 1105 bytes} [Wed Jun 15 14:32:33 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:32:33.402654 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9754dc4-edf7-40c9-81db-a6d7bddc428b None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:33.507616 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9754dc4-edf7-40c9-81db-a6d7bddc428b None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:33.511573 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a297e93-bca6-4fb3-9e56-bb52b131b656 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:33.531841 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:33.546027 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:33.547199 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f95d82a-1161-400d-9004-bc4fb1066c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:33.548871 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f95d82a-1161-400d-9004-bc4fb1066c75 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:33.548988 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f95d82a-1161-400d-9004-bc4fb1066c75 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:33.551411 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f95d82a-1161-400d-9004-bc4fb1066c75 None admin] PATH_INFO: `/v3/services` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:33.551411 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f95d82a-1161-400d-9004-bc4fb1066c75 None admin] RBAC: Adding query filter params (name=image) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:33.551411 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f95d82a-1161-400d-9004-bc4fb1066c75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:33.553972 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f95d82a-1161-400d-9004-bc4fb1066c75 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:33.557220 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dc93be-02dd-494c-a262-8ed9936aa6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:33.558751 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dc93be-02dd-494c-a262-8ed9936aa6bd None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:33.559105 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dc93be-02dd-494c-a262-8ed9936aa6bd None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:33.559454 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dc93be-02dd-494c-a262-8ed9936aa6bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:33.565326 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 516/1037] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:32:33 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:32:33.566156 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f95d82a-1161-400d-9004-bc4fb1066c75 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:33.570192 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dc93be-02dd-494c-a262-8ed9936aa6bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:33.571778 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dc93be-02dd-494c-a262-8ed9936aa6bd None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:33.589756 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-e4dc93be-02dd-494c-a262-8ed9936aa6bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:33.589756 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:33.589756 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:33.589756 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:33.589756 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:33.589756 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:33.589756 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:33.589756 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:33.589756 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:33.589756 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:33.589756 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:33.589756 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:33.589756 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:32:33.589756 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:32:33.589756 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:32:33.590868 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:32:33.590868 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:33.590868 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:33.590868 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:33.590868 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:33.590868 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:33.590868 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:33.590868 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:33.590868 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:33.590868 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:33.590868 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:33.590868 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:33.590868 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:33.590868 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:33.590868 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:33.590868 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:33.590868 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:33.591994 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:32:33.591994 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:32:33.591994 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:33.591994 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:33.593541 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 522/1038] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:32:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:33.598691 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dc93be-02dd-494c-a262-8ed9936aa6bd None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:33.684236 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f95d82a-1161-400d-9004-bc4fb1066c75 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:33.701313 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dc93be-02dd-494c-a262-8ed9936aa6bd None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:33.720459 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e13ce5c-e553-4669-b2ed-5d3f66a32b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:33.721501 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13ce5c-e553-4669-b2ed-5d3f66a32b96 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:33.721929 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13ce5c-e553-4669-b2ed-5d3f66a32b96 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:33.722364 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13ce5c-e553-4669-b2ed-5d3f66a32b96 None admin] PATH_INFO: `/v3/services` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:33.723077 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e13ce5c-e553-4669-b2ed-5d3f66a32b96 None admin] RBAC: Adding query filter params (type=image) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:33.723385 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e13ce5c-e553-4669-b2ed-5d3f66a32b96 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:33.725011 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e13ce5c-e553-4669-b2ed-5d3f66a32b96 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:33.732408 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 517/1039] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:32:33 2022] GET /identity/v3/services?type=image => generated 350 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:32:33.736703 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e13ce5c-e553-4669-b2ed-5d3f66a32b96 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:33.738844 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dee4e9-9c1c-4745-a769-1231de6a1c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:33.739755 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dee4e9-9c1c-4745-a769-1231de6a1c67 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:33.740037 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dee4e9-9c1c-4745-a769-1231de6a1c67 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:33.740347 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dee4e9-9c1c-4745-a769-1231de6a1c67 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:33.741072 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dee4e9-9c1c-4745-a769-1231de6a1c67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:33.741381 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dee4e9-9c1c-4745-a769-1231de6a1c67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:33.742733 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dee4e9-9c1c-4745-a769-1231de6a1c67 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:33.759047 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 523/1040] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:32:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:33.764068 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dee4e9-9c1c-4745-a769-1231de6a1c67 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:33.794634 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:33.808662 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:33.848784 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e13ce5c-e553-4669-b2ed-5d3f66a32b96 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:33.874447 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dee4e9-9c1c-4745-a769-1231de6a1c67 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:33.885110 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a970c3-7717-4c45-96fc-dc40e2429bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:33.885830 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a970c3-7717-4c45-96fc-dc40e2429bd8 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:33.886121 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a970c3-7717-4c45-96fc-dc40e2429bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:33.886410 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a970c3-7717-4c45-96fc-dc40e2429bd8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:33.887178 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a970c3-7717-4c45-96fc-dc40e2429bd8 None admin] RBAC: Adding query filter params (interface=public, service_id=fdec9af8406440d4ba87e8fa596a235a, region_id=RegionOne) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:33.887492 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a970c3-7717-4c45-96fc-dc40e2429bd8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:33.888890 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a970c3-7717-4c45-96fc-dc40e2429bd8 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:33.894165 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 518/1041] 10.222.0.42 () {56 vars in 1251 bytes} [Wed Jun 15 14:32:33 2022] GET /identity/v3/endpoints?service_id=fdec9af8406440d4ba87e8fa596a235a&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:32:33.900519 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a970c3-7717-4c45-96fc-dc40e2429bd8 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:33.910555 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0834c212-4c56-4db2-8b7d-9596c5f86dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:33.911352 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0834c212-4c56-4db2-8b7d-9596c5f86dc5 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:33.911637 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0834c212-4c56-4db2-8b7d-9596c5f86dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:33.911917 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0834c212-4c56-4db2-8b7d-9596c5f86dc5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:33.919957 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0834c212-4c56-4db2-8b7d-9596c5f86dc5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:33.921427 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0834c212-4c56-4db2-8b7d-9596c5f86dc5 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:33.935664 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-0834c212-4c56-4db2-8b7d-9596c5f86dc5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:32:33.935664 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:33.935664 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:33.935664 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:33.935664 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:33.935664 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:33.935664 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:33.935664 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:33.935664 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:33.935664 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:33.935664 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:33.935664 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:33.935664 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:32:33.935664 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:32:33.935664 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:32:33.936606 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:32:33.936606 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:33.936606 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:33.936606 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:33.936606 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:33.936606 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:33.936606 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:33.936606 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:33.936606 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:33.936606 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:33.936606 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:33.936606 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:33.936606 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:33.936606 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:33.936606 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:33.936606 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:33.936606 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:33.938380 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:32:33.938380 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:32:33.938380 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:32:33.938380 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:32:33.938380 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:32:33.938380 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:32:33.938380 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:32:33.938380 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:33.938380 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 524/1042] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jun 15 14:32:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:32:33.942515 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0834c212-4c56-4db2-8b7d-9596c5f86dc5 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:34.012725 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a970c3-7717-4c45-96fc-dc40e2429bd8 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:34.028318 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:34.042179 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:34.046120 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0834c212-4c56-4db2-8b7d-9596c5f86dc5 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:34.050841 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d37c2df-28f0-49b1-bb75-761a31f5165d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:34.051504 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d37c2df-28f0-49b1-bb75-761a31f5165d None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:34.051792 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d37c2df-28f0-49b1-bb75-761a31f5165d None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:34.052071 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d37c2df-28f0-49b1-bb75-761a31f5165d None admin] PATH_INFO: `/v3/services` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:34.052788 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d37c2df-28f0-49b1-bb75-761a31f5165d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:34.054573 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d37c2df-28f0-49b1-bb75-761a31f5165d None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:34.064239 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 519/1043] 10.222.0.42 () {56 vars in 1087 bytes} [Wed Jun 15 14:32:33 2022] GET /identity/v3/services => generated 1957 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:32:34.083858 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e0847e-71c2-496f-9cbe-059cb4367bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:34.084582 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e0847e-71c2-496f-9cbe-059cb4367bca None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:34.084866 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e0847e-71c2-496f-9cbe-059cb4367bca None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:34.085142 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e0847e-71c2-496f-9cbe-059cb4367bca None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:34.085949 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e0847e-71c2-496f-9cbe-059cb4367bca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:34.086305 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e0847e-71c2-496f-9cbe-059cb4367bca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:34.087734 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e0847e-71c2-496f-9cbe-059cb4367bca None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:34.102945 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 525/1044] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jun 15 14:32:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:32:34.109298 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d37c2df-28f0-49b1-bb75-761a31f5165d None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:34.222835 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d37c2df-28f0-49b1-bb75-761a31f5165d None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:34.261546 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbd7e7f-a4ea-46b3-870f-ab65cc606f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:34.262291 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd7e7f-a4ea-46b3-870f-ab65cc606f06 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:34.262580 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd7e7f-a4ea-46b3-870f-ab65cc606f06 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:34.262912 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd7e7f-a4ea-46b3-870f-ab65cc606f06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:34.263687 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbd7e7f-a4ea-46b3-870f-ab65cc606f06 None admin] RBAC: Adding query filter params (role.id=76f33d89f76347be9ef2fdac0435173f, scope.project.id=3094ff8966f149bda784da6a51b1358f, user.id=0927c05c340c48e48b18754888aa9c35) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:34.264003 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbd7e7f-a4ea-46b3-870f-ab65cc606f06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:34.265473 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbd7e7f-a4ea-46b3-870f-ab65cc606f06 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:34.271552 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 520/1045] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jun 15 14:32:34 2022] GET /identity/v3/role_assignments?user.id=0927c05c340c48e48b18754888aa9c35&scope.project.id=3094ff8966f149bda784da6a51b1358f&role.id=76f33d89f76347be9ef2fdac0435173f => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:32:35.155554 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac1ac5-36ba-4cd6-8191-0602942dbf73 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:35.156078 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac1ac5-36ba-4cd6-8191-0602942dbf73 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:35.156184 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac1ac5-36ba-4cd6-8191-0602942dbf73 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:35.157591 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 526/1046] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:32:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:32:35.163221 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48d2a8d-e819-4c8a-803c-18f2287efb28 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:35.163729 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48d2a8d-e819-4c8a-803c-18f2287efb28 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:35.163803 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48d2a8d-e819-4c8a-803c-18f2287efb28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:35.181599 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-a48d2a8d-e819-4c8a-803c-18f2287efb28 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:32:35.183396 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-a48d2a8d-e819-4c8a-803c-18f2287efb28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:32:35.218435 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:35.252518 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 521/1047] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:32:35 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:32:35.274688 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcac1ac5-36ba-4cd6-8191-0602942dbf73 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:35.383933 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcac1ac5-36ba-4cd6-8191-0602942dbf73 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:35.419155 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb260f8-59c3-4f10-9c37-c27fb4a439e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:35.419867 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb260f8-59c3-4f10-9c37-c27fb4a439e7 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:35.420178 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb260f8-59c3-4f10-9c37-c27fb4a439e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:35.420457 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb260f8-59c3-4f10-9c37-c27fb4a439e7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:35.431518 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb260f8-59c3-4f10-9c37-c27fb4a439e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:32:35.439166 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-3eb260f8-59c3-4f10-9c37-c27fb4a439e7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:32:35.439166 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:35.439166 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:35.439166 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:35.439166 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:35.439166 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:35.439166 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:35.439166 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:35.439166 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:35.439166 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:35.439166 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:35.439166 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:35.439166 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:32:35.439166 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:32:35.439166 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:32:35.439166 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:32:35.445161 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:32:35.445161 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:32:35.445161 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:32:35.445161 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:32:35.445161 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:35.445161 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:35.445161 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:35.445161 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:35.445161 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:35.445161 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:35.445161 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:35.445161 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:35.445161 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:35.445161 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:35.445161 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:35.445161 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:35.446314 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:35.446314 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:35.446314 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:35.446314 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:35.446314 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:32:35.446314 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:32:35.446314 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:32:35.446314 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:32:35.446314 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:32:35.446314 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:32:35.446314 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:32:35.446314 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:35.446314 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 527/1048] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jun 15 14:32:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:35.448840 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48d2a8d-e819-4c8a-803c-18f2287efb28 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:35.557176 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48d2a8d-e819-4c8a-803c-18f2287efb28 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:35.592981 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b12235a-17ff-48e6-872b-7f1ba1e0cf65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:35.593652 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b12235a-17ff-48e6-872b-7f1ba1e0cf65 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:35.593939 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b12235a-17ff-48e6-872b-7f1ba1e0cf65 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:35.594219 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b12235a-17ff-48e6-872b-7f1ba1e0cf65 None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:35.594992 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b12235a-17ff-48e6-872b-7f1ba1e0cf65 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:35.595306 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b12235a-17ff-48e6-872b-7f1ba1e0cf65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:35.596702 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b12235a-17ff-48e6-872b-7f1ba1e0cf65 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:35.606012 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 522/1049] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jun 15 14:32:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:35.611637 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb260f8-59c3-4f10-9c37-c27fb4a439e7 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:35.631889 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:35.713445 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb260f8-59c3-4f10-9c37-c27fb4a439e7 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:35.747184 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb157a35-1eb0-492e-93fb-84f0a5a0519c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:35.748060 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb157a35-1eb0-492e-93fb-84f0a5a0519c None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:35.748368 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb157a35-1eb0-492e-93fb-84f0a5a0519c None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:35.748644 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb157a35-1eb0-492e-93fb-84f0a5a0519c None admin] PATH_INFO: `/v3/users/65054510545f44588d9cff7551669fd1` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:35.762547 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb157a35-1eb0-492e-93fb-84f0a5a0519c None admin] RBAC: Authorizing `identity:get_user(user_id=65054510545f44588d9cff7551669fd1)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:35.763986 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb157a35-1eb0-492e-93fb-84f0a5a0519c None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:35.770332 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 528/1050] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jun 15 14:32:35 2022] GET /identity/v3/users/65054510545f44588d9cff7551669fd1 => generated 285 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:35.771862 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b12235a-17ff-48e6-872b-7f1ba1e0cf65 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:35.809956 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:35.876324 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b12235a-17ff-48e6-872b-7f1ba1e0cf65 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:35.909601 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a457278-c06a-415a-b88b-b0f863d9f545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:35.910245 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a457278-c06a-415a-b88b-b0f863d9f545 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:35.910522 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a457278-c06a-415a-b88b-b0f863d9f545 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:35.910793 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a457278-c06a-415a-b88b-b0f863d9f545 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:35.980853 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a457278-c06a-415a-b88b-b0f863d9f545 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:35.982226 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a457278-c06a-415a-b88b-b0f863d9f545 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:36.015728 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:36.083292 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 523/1051] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jun 15 14:32:35 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:32:36.089614 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb157a35-1eb0-492e-93fb-84f0a5a0519c None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:36.173510 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:36.197403 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb157a35-1eb0-492e-93fb-84f0a5a0519c None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:36.208371 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5914b9c2-349a-4e75-8c41-db049aba88f1 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:36.208879 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5914b9c2-349a-4e75-8c41-db049aba88f1 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:36.208959 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5914b9c2-349a-4e75-8c41-db049aba88f1 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:36.210414 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 524/1052] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:32:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:32:36.215403 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a981c331-153d-4250-8c85-ce488de6dd14 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:36.215688 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a981c331-153d-4250-8c85-ce488de6dd14 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:36.215991 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a981c331-153d-4250-8c85-ce488de6dd14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:36.232121 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-a981c331-153d-4250-8c85-ce488de6dd14 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:32:36.233233 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-a981c331-153d-4250-8c85-ce488de6dd14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:32:36.234080 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea5a23d-5003-48a9-917d-a1d982f36d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:36.234947 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea5a23d-5003-48a9-917d-a1d982f36d08 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:36.235237 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea5a23d-5003-48a9-917d-a1d982f36d08 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:36.235515 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea5a23d-5003-48a9-917d-a1d982f36d08 None admin] PATH_INFO: `/v3/projects/127cea9f376b424586558c0990cbf7ef` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:36.236727 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea5a23d-5003-48a9-917d-a1d982f36d08 None admin] RBAC: Authorizing `identity:get_project(project_id=127cea9f376b424586558c0990cbf7ef)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:36.238142 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea5a23d-5003-48a9-917d-a1d982f36d08 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:36.244830 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 529/1053] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jun 15 14:32:36 2022] GET /identity/v3/projects/127cea9f376b424586558c0990cbf7ef => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:36.246509 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea5a23d-5003-48a9-917d-a1d982f36d08 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:36.306629 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 525/1054] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:32:36 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:32:36.330043 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a981c331-153d-4250-8c85-ce488de6dd14 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:36.354029 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea5a23d-5003-48a9-917d-a1d982f36d08 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:36.392262 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670c95a1-4158-47c3-aaba-437e9696d500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:36.392923 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670c95a1-4158-47c3-aaba-437e9696d500 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:36.393211 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670c95a1-4158-47c3-aaba-437e9696d500 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:36.393491 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670c95a1-4158-47c3-aaba-437e9696d500 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:36.394316 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670c95a1-4158-47c3-aaba-437e9696d500 None admin] RBAC: Adding query filter params (role.id=2bd1979f53bb4853a50dd5bee6aa0782, scope.project.id=127cea9f376b424586558c0990cbf7ef, user.id=65054510545f44588d9cff7551669fd1) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:36.394634 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670c95a1-4158-47c3-aaba-437e9696d500 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:36.396045 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670c95a1-4158-47c3-aaba-437e9696d500 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:36.396613 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:36.412119 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 530/1055] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jun 15 14:32:36 2022] GET /identity/v3/role_assignments?user.id=65054510545f44588d9cff7551669fd1&scope.project.id=127cea9f376b424586558c0990cbf7ef&role.id=2bd1979f53bb4853a50dd5bee6aa0782 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:32:36.412119 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92931bed-f9d8-4a1d-987d-781c5c44b2ac None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:36.412119 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92931bed-f9d8-4a1d-987d-781c5c44b2ac None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:36.412119 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92931bed-f9d8-4a1d-987d-781c5c44b2ac None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:36.412119 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 531/1056] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:32:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:32:36.417923 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d54b68e-e436-43c9-a27c-c0effc13ed06 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:36.417923 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d54b68e-e436-43c9-a27c-c0effc13ed06 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:36.417923 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d54b68e-e436-43c9-a27c-c0effc13ed06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:36.436976 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-5d54b68e-e436-43c9-a27c-c0effc13ed06 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:32:36.438568 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-5d54b68e-e436-43c9-a27c-c0effc13ed06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:32:36.441783 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:36.449522 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a981c331-153d-4250-8c85-ce488de6dd14 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:36.488608 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96483b68-a249-4b43-a769-96456f8a598e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:36.489325 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96483b68-a249-4b43-a769-96456f8a598e None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:36.489609 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96483b68-a249-4b43-a769-96456f8a598e None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:36.489887 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96483b68-a249-4b43-a769-96456f8a598e None admin] PATH_INFO: `/v3/services/image` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:36.490550 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96483b68-a249-4b43-a769-96456f8a598e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:36.491953 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96483b68-a249-4b43-a769-96456f8a598e None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:36.504720 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-96483b68-a249-4b43-a769-96456f8a598e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 14:32:36.504720 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:36.504720 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:32:36.504720 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:32:36.504720 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:32:36.504720 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:32:36.504720 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:32:36.504720 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:32:36.504720 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 14:32:36.504720 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:32:36.504720 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:32:36.504720 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:32:36.504720 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:36.504720 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:36.504720 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:36.504720 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:36.504720 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:36.505811 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:36.505811 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:36.505811 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:36.505811 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:36.505811 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:36.505811 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:36.505811 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:32:36.505811 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:32:36.505811 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:32:36.505811 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:32:36.505811 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:36.505811 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:36.505811 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:36.505811 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:36.505811 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:36.505811 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:36.509143 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:36.509143 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:36.509143 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:36.509143 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:36.509143 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:36.509143 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:36.509143 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:36.509143 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:36.509143 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:36.509143 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:36.509143 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:32:36.509143 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:32:36.509143 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 14:32:36.509143 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:32:36.509143 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 526/1057] 10.222.0.42 () {56 vars in 1105 bytes} [Wed Jun 15 14:32:36 2022] GET /identity/v3/services/image => generated 86 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:32:36.512783 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96483b68-a249-4b43-a769-96456f8a598e None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:36.532479 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 532/1058] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:32:36 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:32:36.547873 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d54b68e-e436-43c9-a27c-c0effc13ed06 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:36.633320 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96483b68-a249-4b43-a769-96456f8a598e None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:36.654977 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d54b68e-e436-43c9-a27c-c0effc13ed06 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:36.659653 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:36.668289 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac8e550-6b8b-4cc4-bb83-e6e247a51417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:36.675146 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac8e550-6b8b-4cc4-bb83-e6e247a51417 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:36.675146 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac8e550-6b8b-4cc4-bb83-e6e247a51417 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:36.675146 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac8e550-6b8b-4cc4-bb83-e6e247a51417 None admin] PATH_INFO: `/v3/services` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:36.675146 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac8e550-6b8b-4cc4-bb83-e6e247a51417 None admin] RBAC: Adding query filter params (name=image) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:36.675146 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac8e550-6b8b-4cc4-bb83-e6e247a51417 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:36.675146 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac8e550-6b8b-4cc4-bb83-e6e247a51417 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:36.681480 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 527/1059] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:32:36 2022] GET /identity/v3/services?name=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:32:36.682341 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac8e550-6b8b-4cc4-bb83-e6e247a51417 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:36.697808 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704cf521-b330-4ff4-a302-d052416b0f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:36.698658 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704cf521-b330-4ff4-a302-d052416b0f22 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:36.698939 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704cf521-b330-4ff4-a302-d052416b0f22 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:36.699213 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704cf521-b330-4ff4-a302-d052416b0f22 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:36.706730 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:36.710687 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704cf521-b330-4ff4-a302-d052416b0f22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:32:36.718876 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-704cf521-b330-4ff4-a302-d052416b0f22 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 14:32:36.718876 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:36.718876 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:36.718876 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:36.718876 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:36.718876 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:36.718876 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:36.718876 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:36.718876 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:36.718876 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:36.718876 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:36.718876 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:36.718876 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:32:36.718876 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:32:36.718876 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:32:36.718876 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:32:36.721299 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:32:36.721299 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:32:36.721299 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:32:36.721299 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:32:36.721299 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:36.721299 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:36.721299 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:36.721299 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:36.721299 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:36.721299 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:36.721299 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:36.721299 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:36.721299 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:36.721299 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:36.721299 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:36.721299 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:36.722368 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:36.722368 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:36.722368 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:36.722368 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:36.722368 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:32:36.722368 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:32:36.722368 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:32:36.722368 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:32:36.722368 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:32:36.722368 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:32:36.722368 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 14:32:36.722368 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:36.722368 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 533/1060] 10.222.0.42 () {56 vars in 1120 bytes} [Wed Jun 15 14:32:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:32:36.726410 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704cf521-b330-4ff4-a302-d052416b0f22 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:36.797313 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac8e550-6b8b-4cc4-bb83-e6e247a51417 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:36.833319 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a089def4-eac2-4a06-92b6-7c07a384fd28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:36.833975 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a089def4-eac2-4a06-92b6-7c07a384fd28 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:36.834367 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a089def4-eac2-4a06-92b6-7c07a384fd28 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:36.834643 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a089def4-eac2-4a06-92b6-7c07a384fd28 None admin] PATH_INFO: `/v3/services` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:36.835459 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a089def4-eac2-4a06-92b6-7c07a384fd28 None admin] RBAC: Adding query filter params (type=image) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:36.835766 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a089def4-eac2-4a06-92b6-7c07a384fd28 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:36.837166 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a089def4-eac2-4a06-92b6-7c07a384fd28 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:36.837872 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704cf521-b330-4ff4-a302-d052416b0f22 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:36.847226 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 528/1061] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:32:36 2022] GET /identity/v3/services?type=image => generated 350 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:32:36.848286 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a089def4-eac2-4a06-92b6-7c07a384fd28 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:36.878498 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86140d94-adda-4b09-a326-d98fbb83866c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:36.879170 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86140d94-adda-4b09-a326-d98fbb83866c None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:36.879441 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86140d94-adda-4b09-a326-d98fbb83866c None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:36.879711 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86140d94-adda-4b09-a326-d98fbb83866c None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:36.880480 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86140d94-adda-4b09-a326-d98fbb83866c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:36.880784 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86140d94-adda-4b09-a326-d98fbb83866c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:36.882165 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86140d94-adda-4b09-a326-d98fbb83866c None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:36.890302 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 534/1062] 10.222.0.42 () {56 vars in 1115 bytes} [Wed Jun 15 14:32:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:32:36.895854 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86140d94-adda-4b09-a326-d98fbb83866c None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:36.908400 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:36.956367 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a089def4-eac2-4a06-92b6-7c07a384fd28 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:36.959816 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:36.994349 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3f58ce-e2fa-4ad2-818d-495e206ffa33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:36.995644 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3f58ce-e2fa-4ad2-818d-495e206ffa33 None admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:36.995965 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3f58ce-e2fa-4ad2-818d-495e206ffa33 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:36.996415 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3f58ce-e2fa-4ad2-818d-495e206ffa33 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:36.997004 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3f58ce-e2fa-4ad2-818d-495e206ffa33 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:36.998366 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3f58ce-e2fa-4ad2-818d-495e206ffa33 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:37.003115 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86140d94-adda-4b09-a326-d98fbb83866c None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:37.036387 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb83fe83-3927-4eb7-a48d-de69401f90a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:37.037179 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb83fe83-3927-4eb7-a48d-de69401f90a7 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:37.037457 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb83fe83-3927-4eb7-a48d-de69401f90a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:37.037731 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb83fe83-3927-4eb7-a48d-de69401f90a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:37.048297 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb83fe83-3927-4eb7-a48d-de69401f90a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:37.049651 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb83fe83-3927-4eb7-a48d-de69401f90a7 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:37.068851 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-cb83fe83-3927-4eb7-a48d-de69401f90a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:37.068851 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:37.068851 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:37.068851 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:37.068851 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:37.068851 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:37.068851 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:37.068851 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:37.068851 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:37.068851 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:37.068851 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:37.068851 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:37.068851 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:32:37.068851 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:32:37.068851 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:32:37.071154 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:32:37.071154 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:37.071154 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:37.071154 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:37.071154 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:37.071154 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:37.071154 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:37.071154 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:37.071154 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:37.071154 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:37.071154 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:37.071154 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:37.071154 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:37.071154 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:37.071154 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:37.071154 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:37.071154 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:37.072540 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:32:37.072540 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:32:37.072540 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:37.072540 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:37.072540 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 535/1063] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:32:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:32:37.076800 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 529/1064] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Jun 15 14:32:36 2022] POST /identity/v3/endpoints => generated 327 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:32:37.077404 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb83fe83-3927-4eb7-a48d-de69401f90a7 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:37.186090 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb83fe83-3927-4eb7-a48d-de69401f90a7 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:37.200809 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:37.230228 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5d4b0c-75d9-49bf-b1be-488cc8821c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:37.230898 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d4b0c-75d9-49bf-b1be-488cc8821c2e None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:37.231182 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d4b0c-75d9-49bf-b1be-488cc8821c2e None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:37.231454 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d4b0c-75d9-49bf-b1be-488cc8821c2e None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:37.232199 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5d4b0c-75d9-49bf-b1be-488cc8821c2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:37.232505 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5d4b0c-75d9-49bf-b1be-488cc8821c2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:37.233869 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5d4b0c-75d9-49bf-b1be-488cc8821c2e None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:37.247762 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 536/1065] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:32:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:32:37.252824 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3f58ce-e2fa-4ad2-818d-495e206ffa33 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:37.362414 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3f58ce-e2fa-4ad2-818d-495e206ffa33 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:37.375784 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:37.396635 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51ba8c9-effc-48c4-880c-214b576f563a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:37.397408 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ba8c9-effc-48c4-880c-214b576f563a None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:37.397692 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ba8c9-effc-48c4-880c-214b576f563a None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:37.397968 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ba8c9-effc-48c4-880c-214b576f563a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:37.405735 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51ba8c9-effc-48c4-880c-214b576f563a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:37.407160 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51ba8c9-effc-48c4-880c-214b576f563a None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:37.421999 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-b51ba8c9-effc-48c4-880c-214b576f563a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:32:37.421999 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:37.421999 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:37.421999 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:37.421999 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:37.421999 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:37.421999 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:37.421999 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:37.421999 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:37.421999 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:37.421999 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:37.421999 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:37.421999 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:32:37.421999 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:32:37.421999 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:32:37.421999 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:32:37.424351 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:37.424351 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:37.424351 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:32:37.424351 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:32:37.424351 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:32:37.424351 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:32:37.424351 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:37.424351 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:37.424351 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:37.424351 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:37.424351 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:37.424351 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:37.424351 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:37.424351 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:37.424351 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:37.424351 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:37.424351 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:37.424351 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:37.425320 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:37.425320 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:37.425320 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:32:37.425320 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:32:37.425320 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:32:37.425320 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:32:37.425320 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:32:37.425320 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:32:37.425320 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:32:37.425320 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:32:37.425320 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 530/1066] 10.222.0.42 () {56 vars in 1093 bytes} [Wed Jun 15 14:32:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:37.428948 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5d4b0c-75d9-49bf-b1be-488cc8821c2e None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:37.533437 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5d4b0c-75d9-49bf-b1be-488cc8821c2e None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:37.574878 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d289184-0648-4214-9ee0-90d027efbb6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:37.575553 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d289184-0648-4214-9ee0-90d027efbb6e None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:37.575832 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d289184-0648-4214-9ee0-90d027efbb6e None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:37.576126 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d289184-0648-4214-9ee0-90d027efbb6e None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:37.576970 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d289184-0648-4214-9ee0-90d027efbb6e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:37.577283 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d289184-0648-4214-9ee0-90d027efbb6e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:37.578709 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d289184-0648-4214-9ee0-90d027efbb6e None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:37.601806 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 537/1067] 10.222.0.42 () {56 vars in 1133 bytes} [Wed Jun 15 14:32:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:37.603075 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51ba8c9-effc-48c4-880c-214b576f563a None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:37.707956 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51ba8c9-effc-48c4-880c-214b576f563a None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:37.740174 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034bea24-e230-48c2-9ed2-18ea16eb5bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:37.740958 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034bea24-e230-48c2-9ed2-18ea16eb5bc3 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:37.741236 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034bea24-e230-48c2-9ed2-18ea16eb5bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:37.741510 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034bea24-e230-48c2-9ed2-18ea16eb5bc3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:37.752621 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034bea24-e230-48c2-9ed2-18ea16eb5bc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:37.754044 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034bea24-e230-48c2-9ed2-18ea16eb5bc3 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:37.771681 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-034bea24-e230-48c2-9ed2-18ea16eb5bc3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:37.771681 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:37.771681 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:37.771681 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:37.771681 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:37.771681 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:37.771681 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:37.771681 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:37.771681 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:37.771681 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:37.771681 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:37.771681 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:37.771681 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:32:37.771681 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:32:37.771681 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:32:37.772628 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:32:37.772628 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:37.772628 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:37.772628 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:37.772628 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:37.772628 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:37.772628 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:37.772628 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:37.772628 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:37.772628 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:37.772628 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:37.772628 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:37.772628 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:37.772628 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:37.772628 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:37.772628 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:37.772628 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:37.773795 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:32:37.773795 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:32:37.773795 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:37.773795 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:32:37.774556 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 531/1068] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:32:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:37.779782 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d289184-0648-4214-9ee0-90d027efbb6e None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:37.812349 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:37.883911 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d289184-0648-4214-9ee0-90d027efbb6e None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:37.918779 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c5fd1b-198a-4d2f-8bc4-afe9afac07d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:37.919439 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c5fd1b-198a-4d2f-8bc4-afe9afac07d4 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:37.919721 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c5fd1b-198a-4d2f-8bc4-afe9afac07d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:37.919999 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c5fd1b-198a-4d2f-8bc4-afe9afac07d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:37.920744 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c5fd1b-198a-4d2f-8bc4-afe9afac07d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:37.921054 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c5fd1b-198a-4d2f-8bc4-afe9afac07d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:37.922442 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c5fd1b-198a-4d2f-8bc4-afe9afac07d4 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:37.938610 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 538/1069] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:32:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:37.943743 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034bea24-e230-48c2-9ed2-18ea16eb5bc3 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:37.967961 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:38.044059 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034bea24-e230-48c2-9ed2-18ea16eb5bc3 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:38.078568 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e6ca09-b607-4266-80cb-c6a75e37af37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:38.079358 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e6ca09-b607-4266-80cb-c6a75e37af37 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:38.079642 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e6ca09-b607-4266-80cb-c6a75e37af37 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:38.079918 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e6ca09-b607-4266-80cb-c6a75e37af37 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:38.086567 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e6ca09-b607-4266-80cb-c6a75e37af37 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:38.087979 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e6ca09-b607-4266-80cb-c6a75e37af37 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:38.107053 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-08e6ca09-b607-4266-80cb-c6a75e37af37 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:32:38.107053 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:38.107053 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:38.107053 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:38.107053 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:38.107053 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:38.107053 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:38.107053 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:38.107053 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:38.107053 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:38.107053 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:38.107053 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:38.107053 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:32:38.107053 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:32:38.107053 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:32:38.108005 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:32:38.108005 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:38.108005 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:38.108005 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:38.108005 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:38.108005 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:38.108005 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:38.108005 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:38.108005 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:38.108005 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:38.108005 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:38.108005 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:38.108005 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:38.108005 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:38.108005 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:38.108005 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:38.108005 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:38.108945 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:32:38.108945 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:32:38.108945 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:32:38.108945 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:32:38.108945 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:32:38.108945 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:32:38.108945 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:32:38.108945 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:32:38.109956 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 532/1070] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jun 15 14:32:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:38.113978 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c5fd1b-198a-4d2f-8bc4-afe9afac07d4 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:38.205861 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:38.221390 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c5fd1b-198a-4d2f-8bc4-afe9afac07d4 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:38.255998 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e28dd0-7f7d-475f-bc2f-3bb24e40a856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:38.256684 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e28dd0-7f7d-475f-bc2f-3bb24e40a856 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:38.256969 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e28dd0-7f7d-475f-bc2f-3bb24e40a856 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:38.257241 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e28dd0-7f7d-475f-bc2f-3bb24e40a856 None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:38.258017 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e28dd0-7f7d-475f-bc2f-3bb24e40a856 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:38.258324 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e28dd0-7f7d-475f-bc2f-3bb24e40a856 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:38.259739 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e28dd0-7f7d-475f-bc2f-3bb24e40a856 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:38.278185 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 539/1071] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jun 15 14:32:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:38.279526 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e6ca09-b607-4266-80cb-c6a75e37af37 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:38.366377 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:38.383175 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e6ca09-b607-4266-80cb-c6a75e37af37 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:38.418193 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d49873-f17a-4621-b460-b213865fc8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:38.419059 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d49873-f17a-4621-b460-b213865fc8d0 None admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:38.419335 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d49873-f17a-4621-b460-b213865fc8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:38.419604 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d49873-f17a-4621-b460-b213865fc8d0 None admin] PATH_INFO: `/v3/projects/3094ff8966f149bda784da6a51b1358f/users/0927c05c340c48e48b18754888aa9c35/roles/76f33d89f76347be9ef2fdac0435173f` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:38.421756 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d49873-f17a-4621-b460-b213865fc8d0 None admin] RBAC: Authorizing `identity:create_grant(project_id=3094ff8966f149bda784da6a51b1358f, user_id=0927c05c340c48e48b18754888aa9c35, role_id=76f33d89f76347be9ef2fdac0435173f)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:38.423181 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d49873-f17a-4621-b460-b213865fc8d0 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:38.496295 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 533/1072] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Jun 15 14:32:38 2022] PUT /identity/v3/projects/3094ff8966f149bda784da6a51b1358f/users/0927c05c340c48e48b18754888aa9c35/roles/76f33d89f76347be9ef2fdac0435173f => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:32:38.571002 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655d85b4-7782-41a5-836d-1086a2978ec0 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:38.571312 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655d85b4-7782-41a5-836d-1086a2978ec0 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:38.571574 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655d85b4-7782-41a5-836d-1086a2978ec0 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:38.573197 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 540/1073] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:32:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:32:38.579184 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3d41d5-a300-4807-86b1-3c7d65ae28dd None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:38.579505 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3d41d5-a300-4807-86b1-3c7d65ae28dd None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:38.579752 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3d41d5-a300-4807-86b1-3c7d65ae28dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:38.598103 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-aa3d41d5-a300-4807-86b1-3c7d65ae28dd None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:32:38.599413 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-aa3d41d5-a300-4807-86b1-3c7d65ae28dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:32:38.678543 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 534/1074] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:32:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:32:38.703067 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655d85b4-7782-41a5-836d-1086a2978ec0 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:38.808394 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655d85b4-7782-41a5-836d-1086a2978ec0 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:38.843190 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d339ac35-7b20-4d98-81d0-8b84f7565221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:38.843986 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d339ac35-7b20-4d98-81d0-8b84f7565221 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:38.844315 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d339ac35-7b20-4d98-81d0-8b84f7565221 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:38.844596 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d339ac35-7b20-4d98-81d0-8b84f7565221 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:38.855681 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d339ac35-7b20-4d98-81d0-8b84f7565221 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:32:38.863546 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-d339ac35-7b20-4d98-81d0-8b84f7565221 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:32:38.863546 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:38.863546 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:38.863546 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:38.863546 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:38.863546 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:38.863546 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:38.863546 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:38.863546 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:38.863546 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:38.863546 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:38.863546 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:38.863546 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:32:38.863546 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:32:38.863546 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:32:38.863546 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:32:38.864697 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:32:38.864697 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:32:38.864697 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:32:38.864697 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:32:38.864697 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:38.864697 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:38.864697 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:38.864697 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:38.864697 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:38.864697 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:38.864697 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:38.864697 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:38.864697 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:38.864697 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:38.864697 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:38.864697 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:38.865549 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:38.865549 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:38.865549 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:38.865549 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:38.865549 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:32:38.865549 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:32:38.865549 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:32:38.865549 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:32:38.865549 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:32:38.865549 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:32:38.865549 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:32:38.865549 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:38.865549 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 541/1075] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jun 15 14:32:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:38.873460 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3d41d5-a300-4807-86b1-3c7d65ae28dd None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:38.949970 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:38.975345 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3d41d5-a300-4807-86b1-3c7d65ae28dd None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:39.011149 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b6a5a-8745-4b6b-b887-744cf5d9291a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:39.011928 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b6a5a-8745-4b6b-b887-744cf5d9291a None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:39.012235 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b6a5a-8745-4b6b-b887-744cf5d9291a None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:39.012511 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b6a5a-8745-4b6b-b887-744cf5d9291a None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:39.013247 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104b6a5a-8745-4b6b-b887-744cf5d9291a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:39.013552 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104b6a5a-8745-4b6b-b887-744cf5d9291a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:39.015232 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104b6a5a-8745-4b6b-b887-744cf5d9291a None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:39.028956 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 535/1076] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jun 15 14:32:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:39.030569 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d339ac35-7b20-4d98-81d0-8b84f7565221 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:39.040621 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:39.134236 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d339ac35-7b20-4d98-81d0-8b84f7565221 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:39.169560 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec2634e-ab8e-4aaf-a645-c82c40cb0e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:39.170429 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec2634e-ab8e-4aaf-a645-c82c40cb0e39 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:39.170714 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec2634e-ab8e-4aaf-a645-c82c40cb0e39 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:39.170990 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec2634e-ab8e-4aaf-a645-c82c40cb0e39 None admin] PATH_INFO: `/v3/users/65054510545f44588d9cff7551669fd1` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:39.172287 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec2634e-ab8e-4aaf-a645-c82c40cb0e39 None admin] RBAC: Authorizing `identity:get_user(user_id=65054510545f44588d9cff7551669fd1)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:39.173716 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec2634e-ab8e-4aaf-a645-c82c40cb0e39 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:39.180176 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 542/1077] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jun 15 14:32:39 2022] GET /identity/v3/users/65054510545f44588d9cff7551669fd1 => generated 285 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:39.181604 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b6a5a-8745-4b6b-b887-744cf5d9291a None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:39.255863 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3ac456-6b50-4da6-9f4c-e3b0e72befd2 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:39.256420 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3ac456-6b50-4da6-9f4c-e3b0e72befd2 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:39.256420 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3ac456-6b50-4da6-9f4c-e3b0e72befd2 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:39.257984 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 543/1078] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:32:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:32:39.263393 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8530d14f-6a1c-4b3c-8bc5-fe5593560d9e None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:39.263684 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8530d14f-6a1c-4b3c-8bc5-fe5593560d9e None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:39.263966 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8530d14f-6a1c-4b3c-8bc5-fe5593560d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:39.281738 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-8530d14f-6a1c-4b3c-8bc5-fe5593560d9e None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:32:39.282935 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-8530d14f-6a1c-4b3c-8bc5-fe5593560d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:32:39.292392 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b6a5a-8745-4b6b-b887-744cf5d9291a None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:39.329844 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770e6702-f8f9-40ab-82e6-f8b11b6a1297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:39.330693 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e6702-f8f9-40ab-82e6-f8b11b6a1297 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:39.330976 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e6702-f8f9-40ab-82e6-f8b11b6a1297 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:39.331252 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e6702-f8f9-40ab-82e6-f8b11b6a1297 None admin] PATH_INFO: `/v3/projects/127cea9f376b424586558c0990cbf7ef` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:39.332440 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770e6702-f8f9-40ab-82e6-f8b11b6a1297 None admin] RBAC: Authorizing `identity:get_project(project_id=127cea9f376b424586558c0990cbf7ef)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:39.333853 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770e6702-f8f9-40ab-82e6-f8b11b6a1297 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:39.335528 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 536/1079] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jun 15 14:32:39 2022] GET /identity/v3/projects/127cea9f376b424586558c0990cbf7ef => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:39.341336 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770e6702-f8f9-40ab-82e6-f8b11b6a1297 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:39.344711 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:39.368108 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 544/1080] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:32:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:32:39.383614 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8530d14f-6a1c-4b3c-8bc5-fe5593560d9e None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:39.387864 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:39.449482 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770e6702-f8f9-40ab-82e6-f8b11b6a1297 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:39.486551 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0678eb-8b23-4544-921a-413293e4f446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:39.487430 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0678eb-8b23-4544-921a-413293e4f446 None admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:39.487710 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0678eb-8b23-4544-921a-413293e4f446 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:39.487997 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0678eb-8b23-4544-921a-413293e4f446 None admin] PATH_INFO: `/v3/projects/127cea9f376b424586558c0990cbf7ef/users/65054510545f44588d9cff7551669fd1/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:39.489957 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0678eb-8b23-4544-921a-413293e4f446 None admin] RBAC: Authorizing `identity:create_grant(project_id=127cea9f376b424586558c0990cbf7ef, user_id=65054510545f44588d9cff7551669fd1, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:39.491382 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0678eb-8b23-4544-921a-413293e4f446 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:39.493046 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8530d14f-6a1c-4b3c-8bc5-fe5593560d9e None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:39.528525 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6562170f-8fa2-49fa-9eab-5ac552a61949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:39.529166 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6562170f-8fa2-49fa-9eab-5ac552a61949 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:39.529448 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6562170f-8fa2-49fa-9eab-5ac552a61949 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:39.529718 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6562170f-8fa2-49fa-9eab-5ac552a61949 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:39.565544 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 537/1081] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Jun 15 14:32:39 2022] PUT /identity/v3/projects/127cea9f376b424586558c0990cbf7ef/users/65054510545f44588d9cff7551669fd1/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:32:39.607040 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6562170f-8fa2-49fa-9eab-5ac552a61949 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:39.608538 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6562170f-8fa2-49fa-9eab-5ac552a61949 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:39.612500 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:39.726645 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 545/1082] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jun 15 14:32:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 15 14:32:39.733249 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0678eb-8b23-4544-921a-413293e4f446 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:39.798451 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:39.836582 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0678eb-8b23-4544-921a-413293e4f446 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:39.871431 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d742f262-aeca-425c-b89f-1c2114e5cb0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:39.872244 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d742f262-aeca-425c-b89f-1c2114e5cb0d None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:39.872529 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d742f262-aeca-425c-b89f-1c2114e5cb0d None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:39.872799 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d742f262-aeca-425c-b89f-1c2114e5cb0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:39.884106 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d742f262-aeca-425c-b89f-1c2114e5cb0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:39.885574 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d742f262-aeca-425c-b89f-1c2114e5cb0d None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:39.903240 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-d742f262-aeca-425c-b89f-1c2114e5cb0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:39.903240 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:39.903240 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:39.903240 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:39.903240 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:39.903240 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:39.903240 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:39.903240 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:39.903240 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:39.903240 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:39.903240 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:39.903240 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:39.903240 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:32:39.903240 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:32:39.903240 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:32:39.904417 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:32:39.904417 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:39.904417 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:39.904417 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:39.904417 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:39.904417 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:39.904417 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:39.904417 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:39.904417 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:39.904417 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:39.904417 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:39.904417 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:39.904417 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:39.904417 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:39.904417 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:39.904417 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:39.904417 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:39.905316 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:32:39.905316 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:32:39.905316 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:39.905316 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:32:39.905316 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 538/1083] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:32:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:39.911430 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6562170f-8fa2-49fa-9eab-5ac552a61949 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:40.009817 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6562170f-8fa2-49fa-9eab-5ac552a61949 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:40.012175 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:40.047157 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da358e73-8482-49bc-bf83-f26f7b4e5502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:40.047847 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da358e73-8482-49bc-bf83-f26f7b4e5502 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:40.048155 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da358e73-8482-49bc-bf83-f26f7b4e5502 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:40.048433 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da358e73-8482-49bc-bf83-f26f7b4e5502 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:40.049148 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da358e73-8482-49bc-bf83-f26f7b4e5502 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:40.049453 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da358e73-8482-49bc-bf83-f26f7b4e5502 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:40.050797 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da358e73-8482-49bc-bf83-f26f7b4e5502 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:40.062847 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 546/1084] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:32:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:32:40.654171 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0e31f5-17e9-4049-b8f1-f5f69f343648 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:40.654752 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0e31f5-17e9-4049-b8f1-f5f69f343648 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:40.654752 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0e31f5-17e9-4049-b8f1-f5f69f343648 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:40.656130 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 539/1085] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:32:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:32:40.661894 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e37221-717b-4364-b962-17b30c598388 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:40.662415 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e37221-717b-4364-b962-17b30c598388 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:40.662530 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e37221-717b-4364-b962-17b30c598388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:40.679511 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-88e37221-717b-4364-b962-17b30c598388 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:32:40.680725 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-88e37221-717b-4364-b962-17b30c598388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:32:40.746479 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 547/1086] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:32:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:32:40.769734 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0e31f5-17e9-4049-b8f1-f5f69f343648 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:40.870776 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0e31f5-17e9-4049-b8f1-f5f69f343648 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:40.886032 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:40.903780 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8157011-27d4-41bc-819e-bee78025add0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:40.904608 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8157011-27d4-41bc-819e-bee78025add0 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:40.904892 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8157011-27d4-41bc-819e-bee78025add0 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:40.905166 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8157011-27d4-41bc-819e-bee78025add0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:40.915132 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8157011-27d4-41bc-819e-bee78025add0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:32:40.923919 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-f8157011-27d4-41bc-819e-bee78025add0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 14:32:40.923919 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:40.923919 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:40.923919 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:40.923919 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:40.923919 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:40.923919 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:40.923919 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:40.923919 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:40.923919 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:40.923919 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:40.923919 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:40.923919 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:32:40.923919 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:32:40.923919 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:32:40.923919 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:32:40.925049 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:32:40.925049 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:32:40.925049 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:32:40.925049 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:32:40.925049 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:40.925049 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:40.925049 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:40.925049 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:40.925049 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:40.925049 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:40.925049 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:40.925049 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:40.925049 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:40.925049 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:40.925049 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:40.925049 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:40.926079 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:40.926079 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:40.926079 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:40.926079 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:40.926079 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:32:40.926079 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:32:40.926079 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:32:40.926079 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:32:40.926079 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:32:40.926079 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:32:40.926079 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 14:32:40.926079 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:32:40.926079 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 540/1087] 10.222.0.42 () {56 vars in 1120 bytes} [Wed Jun 15 14:32:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:40.931012 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e37221-717b-4364-b962-17b30c598388 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:41.021986 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:41.029537 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e37221-717b-4364-b962-17b30c598388 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:41.062344 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e429bb-b043-47bd-b3c4-09791f1050b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:41.063000 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e429bb-b043-47bd-b3c4-09791f1050b3 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:41.063280 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e429bb-b043-47bd-b3c4-09791f1050b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:41.063555 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e429bb-b043-47bd-b3c4-09791f1050b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:41.064320 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e429bb-b043-47bd-b3c4-09791f1050b3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:41.064628 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e429bb-b043-47bd-b3c4-09791f1050b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:41.065985 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e429bb-b043-47bd-b3c4-09791f1050b3 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:41.074205 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 548/1088] 10.222.0.42 () {56 vars in 1115 bytes} [Wed Jun 15 14:32:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:41.079869 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8157011-27d4-41bc-819e-bee78025add0 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:41.182728 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8157011-27d4-41bc-819e-bee78025add0 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:41.215980 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3debca-a5c9-4cb7-b57b-186a271ac7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:41.216785 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3debca-a5c9-4cb7-b57b-186a271ac7be None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:41.217061 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3debca-a5c9-4cb7-b57b-186a271ac7be None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:41.217330 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3debca-a5c9-4cb7-b57b-186a271ac7be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:41.227002 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3debca-a5c9-4cb7-b57b-186a271ac7be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:41.228445 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3debca-a5c9-4cb7-b57b-186a271ac7be None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:41.245286 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-6c3debca-a5c9-4cb7-b57b-186a271ac7be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:41.245286 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:41.245286 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:41.245286 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:41.245286 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:41.245286 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:41.245286 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:41.245286 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:41.245286 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:41.245286 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:41.245286 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:41.245286 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:41.245286 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:32:41.245286 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:32:41.245286 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:32:41.246299 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:32:41.246299 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:41.246299 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:41.246299 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:41.246299 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:41.246299 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:41.246299 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:41.246299 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:41.246299 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:41.246299 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:41.246299 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:41.246299 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:41.246299 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:41.246299 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:41.246299 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:41.246299 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:41.246299 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:41.247202 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:32:41.247202 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:32:41.247202 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:41.247202 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:32:41.248230 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 541/1089] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:32:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:41.252019 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e429bb-b043-47bd-b3c4-09791f1050b3 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:41.349037 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e429bb-b043-47bd-b3c4-09791f1050b3 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:41.384001 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d5e3e4-3fab-42ff-9dc1-a730ae860f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:41.384680 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d5e3e4-3fab-42ff-9dc1-a730ae860f27 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:41.384961 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d5e3e4-3fab-42ff-9dc1-a730ae860f27 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:41.385236 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d5e3e4-3fab-42ff-9dc1-a730ae860f27 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:41.386702 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d5e3e4-3fab-42ff-9dc1-a730ae860f27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:41.386926 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d5e3e4-3fab-42ff-9dc1-a730ae860f27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:41.388292 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d5e3e4-3fab-42ff-9dc1-a730ae860f27 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:41.399817 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 549/1090] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:32:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:41.405562 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3debca-a5c9-4cb7-b57b-186a271ac7be None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:41.434473 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:41.504567 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3debca-a5c9-4cb7-b57b-186a271ac7be None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:41.538099 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ed11da-796c-43c9-8c72-35b19800cff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:41.538891 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed11da-796c-43c9-8c72-35b19800cff1 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:41.539174 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed11da-796c-43c9-8c72-35b19800cff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:41.539457 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed11da-796c-43c9-8c72-35b19800cff1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:41.548631 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ed11da-796c-43c9-8c72-35b19800cff1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:41.550076 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ed11da-796c-43c9-8c72-35b19800cff1 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:41.565854 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-78ed11da-796c-43c9-8c72-35b19800cff1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:32:41.565854 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:41.565854 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:41.565854 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:41.565854 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:41.565854 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:41.565854 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:41.565854 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:41.565854 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:41.565854 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:41.565854 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:41.565854 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:41.565854 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:32:41.565854 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:32:41.565854 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:32:41.565854 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:32:41.566985 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:41.566985 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:41.566985 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:32:41.566985 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:32:41.566985 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:32:41.566985 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:32:41.566985 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:41.566985 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:41.566985 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:41.566985 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:41.566985 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:41.566985 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:41.566985 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:41.566985 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:41.566985 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:41.566985 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:41.566985 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:41.566985 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:41.567946 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:41.567946 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:41.567946 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:32:41.567946 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:32:41.567946 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:32:41.567946 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:32:41.567946 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:32:41.567946 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:32:41.567946 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:32:41.567946 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:32:41.567946 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 542/1091] 10.222.0.42 () {56 vars in 1093 bytes} [Wed Jun 15 14:32:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:32:41.572832 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d5e3e4-3fab-42ff-9dc1-a730ae860f27 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:41.591069 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:41.676750 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d5e3e4-3fab-42ff-9dc1-a730ae860f27 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:41.711541 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c509a682-13af-459a-889d-2c3d38344c70 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:41.712036 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3316f624-5048-41f0-b33c-dccb349dd29d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:41.712172 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c509a682-13af-459a-889d-2c3d38344c70 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:41.712172 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c509a682-13af-459a-889d-2c3d38344c70 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:41.712358 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3316f624-5048-41f0-b33c-dccb349dd29d None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:41.712635 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3316f624-5048-41f0-b33c-dccb349dd29d None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:41.712914 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3316f624-5048-41f0-b33c-dccb349dd29d None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:41.713393 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 543/1092] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:32:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:32:41.713811 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3316f624-5048-41f0-b33c-dccb349dd29d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:41.714253 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3316f624-5048-41f0-b33c-dccb349dd29d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:41.715783 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3316f624-5048-41f0-b33c-dccb349dd29d None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:41.718529 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe17fc2-0e8c-4565-aca5-50d0d181c19a None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:41.718814 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe17fc2-0e8c-4565-aca5-50d0d181c19a None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:41.719098 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe17fc2-0e8c-4565-aca5-50d0d181c19a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:41.739455 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 550/1093] 10.222.0.42 () {56 vars in 1133 bytes} [Wed Jun 15 14:32:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:41.739641 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-9fe17fc2-0e8c-4565-aca5-50d0d181c19a None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:32:41.739641 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-9fe17fc2-0e8c-4565-aca5-50d0d181c19a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:32:41.740702 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3316f624-5048-41f0-b33c-dccb349dd29d None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:41.803216 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 544/1094] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:32:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:32:41.814007 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:41.826880 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe17fc2-0e8c-4565-aca5-50d0d181c19a None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:41.837961 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3316f624-5048-41f0-b33c-dccb349dd29d None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:41.844144 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:41.873014 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6bd641-d90b-4bc6-a25e-17bd95bbd6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:41.873683 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6bd641-d90b-4bc6-a25e-17bd95bbd6a4 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:41.873964 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6bd641-d90b-4bc6-a25e-17bd95bbd6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:41.874258 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6bd641-d90b-4bc6-a25e-17bd95bbd6a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:41.927243 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe17fc2-0e8c-4565-aca5-50d0d181c19a None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:41.940078 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6bd641-d90b-4bc6-a25e-17bd95bbd6a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:41.950423 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6bd641-d90b-4bc6-a25e-17bd95bbd6a4 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:41.961354 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db08a7a-7549-4256-a951-12324e41e459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:41.962265 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db08a7a-7549-4256-a951-12324e41e459 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:41.962497 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db08a7a-7549-4256-a951-12324e41e459 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:41.962785 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db08a7a-7549-4256-a951-12324e41e459 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:41.972780 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db08a7a-7549-4256-a951-12324e41e459 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:32:41.980866 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-6db08a7a-7549-4256-a951-12324e41e459 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:32:41.980866 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:41.980866 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:41.980866 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:41.980866 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:41.980866 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:41.980866 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:41.980866 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:41.980866 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:41.980866 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:41.980866 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:41.980866 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:41.980866 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:32:41.980866 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:32:41.980866 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:32:41.980866 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:32:41.982014 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:32:41.982014 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:32:41.982014 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:32:41.982014 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:32:41.982014 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:41.982014 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:41.982014 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:41.982014 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:41.982014 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:41.982014 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:41.982014 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:41.982014 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:41.982014 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:41.982014 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:41.982014 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:41.982014 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:41.983079 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:41.983079 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:41.983079 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:41.983079 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:41.983079 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:32:41.983079 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:32:41.983079 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:32:41.983079 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:32:41.983079 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:32:41.983079 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:32:41.983079 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:32:41.983079 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:32:41.983079 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 545/1095] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jun 15 14:32:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:32:41.987807 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db08a7a-7549-4256-a951-12324e41e459 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:42.017459 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:42.043039 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 551/1096] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jun 15 14:32:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:32:42.048616 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6bd641-d90b-4bc6-a25e-17bd95bbd6a4 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:42.071379 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:42.092270 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db08a7a-7549-4256-a951-12324e41e459 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:42.125415 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fd642b-1161-4d42-a52e-cfadef8feb4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:42.126068 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fd642b-1161-4d42-a52e-cfadef8feb4f None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:42.126345 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fd642b-1161-4d42-a52e-cfadef8feb4f None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:42.126618 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fd642b-1161-4d42-a52e-cfadef8feb4f None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:42.127343 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fd642b-1161-4d42-a52e-cfadef8feb4f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:42.127647 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fd642b-1161-4d42-a52e-cfadef8feb4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:42.129025 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fd642b-1161-4d42-a52e-cfadef8feb4f None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:42.136877 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 546/1097] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jun 15 14:32:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:32:42.142612 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fd642b-1161-4d42-a52e-cfadef8feb4f None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:42.147541 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6bd641-d90b-4bc6-a25e-17bd95bbd6a4 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:42.182979 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d99419-760d-4c7e-8a75-0f144aaa436e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:42.183768 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d99419-760d-4c7e-8a75-0f144aaa436e None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:42.184052 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d99419-760d-4c7e-8a75-0f144aaa436e None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:42.184365 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d99419-760d-4c7e-8a75-0f144aaa436e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:42.194099 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d99419-760d-4c7e-8a75-0f144aaa436e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:42.195528 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d99419-760d-4c7e-8a75-0f144aaa436e None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:42.213923 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-b5d99419-760d-4c7e-8a75-0f144aaa436e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:42.213923 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:42.213923 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:42.213923 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:42.213923 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:42.213923 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:42.213923 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:42.213923 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:42.213923 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:42.213923 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:42.213923 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:42.213923 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:42.213923 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:32:42.213923 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:32:42.213923 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:32:42.214919 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:32:42.214919 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:42.214919 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:42.214919 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:42.214919 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:42.214919 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:42.214919 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:42.214919 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:42.214919 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:42.214919 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:42.214919 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:42.214919 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:42.214919 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:42.214919 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:42.214919 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:42.214919 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:42.214919 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:42.215816 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:32:42.215816 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:32:42.215816 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:42.215816 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:42.215816 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 552/1098] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:32:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:32:42.220657 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d99419-760d-4c7e-8a75-0f144aaa436e None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:42.240643 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fd642b-1161-4d42-a52e-cfadef8feb4f None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:42.247530 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:42.274827 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e726b294-b9ef-4118-a450-dc79e09dfca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:42.275692 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e726b294-b9ef-4118-a450-dc79e09dfca8 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:42.275972 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e726b294-b9ef-4118-a450-dc79e09dfca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:42.276835 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e726b294-b9ef-4118-a450-dc79e09dfca8 None admin] PATH_INFO: `/v3/users/65054510545f44588d9cff7551669fd1` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:42.278157 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e726b294-b9ef-4118-a450-dc79e09dfca8 None admin] RBAC: Authorizing `identity:get_user(user_id=65054510545f44588d9cff7551669fd1)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:42.279573 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e726b294-b9ef-4118-a450-dc79e09dfca8 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:42.281230 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 547/1099] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jun 15 14:32:42 2022] GET /identity/v3/users/65054510545f44588d9cff7551669fd1 => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:32:42.286689 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e726b294-b9ef-4118-a450-dc79e09dfca8 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:42.297922 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:42.320817 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d99419-760d-4c7e-8a75-0f144aaa436e None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:42.354315 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c634e0-00b5-495b-a7c1-b9635a808663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:42.355488 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c634e0-00b5-495b-a7c1-b9635a808663 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:42.355769 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c634e0-00b5-495b-a7c1-b9635a808663 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:42.356053 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c634e0-00b5-495b-a7c1-b9635a808663 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:42.356795 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c634e0-00b5-495b-a7c1-b9635a808663 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:42.357103 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c634e0-00b5-495b-a7c1-b9635a808663 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:42.358484 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c634e0-00b5-495b-a7c1-b9635a808663 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:42.370358 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 553/1100] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:32:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:32:42.375745 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c634e0-00b5-495b-a7c1-b9635a808663 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:42.386497 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e726b294-b9ef-4118-a450-dc79e09dfca8 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:42.420545 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fb1d3c-5b96-4348-8a93-466bd0d0e311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:42.421188 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fb1d3c-5b96-4348-8a93-466bd0d0e311 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:42.421468 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fb1d3c-5b96-4348-8a93-466bd0d0e311 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:42.421738 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fb1d3c-5b96-4348-8a93-466bd0d0e311 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:42.459481 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:42.474383 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c634e0-00b5-495b-a7c1-b9635a808663 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:42.487919 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fb1d3c-5b96-4348-8a93-466bd0d0e311 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:42.489392 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fb1d3c-5b96-4348-8a93-466bd0d0e311 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:42.498873 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:42.507366 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765db5cb-1a29-40d9-aead-757840b08f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:42.511355 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765db5cb-1a29-40d9-aead-757840b08f92 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:42.511355 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765db5cb-1a29-40d9-aead-757840b08f92 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:42.511355 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765db5cb-1a29-40d9-aead-757840b08f92 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:42.518238 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765db5cb-1a29-40d9-aead-757840b08f92 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:42.520349 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765db5cb-1a29-40d9-aead-757840b08f92 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:42.535111 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-765db5cb-1a29-40d9-aead-757840b08f92 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:32:42.535111 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:42.535111 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:42.535111 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:42.535111 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:42.535111 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:42.535111 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:42.535111 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:42.535111 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:42.535111 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:42.535111 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:42.535111 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:42.535111 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:32:42.535111 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:32:42.535111 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:32:42.536181 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:32:42.536181 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:42.536181 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:42.536181 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:42.536181 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:42.536181 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:42.536181 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:42.536181 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:42.536181 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:42.536181 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:42.536181 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:42.536181 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:42.536181 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:42.536181 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:42.536181 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:42.536181 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:42.536181 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:42.537532 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:32:42.537532 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:32:42.537532 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:32:42.537532 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:32:42.537532 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:32:42.537532 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:32:42.537532 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:32:42.537532 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:32:42.537532 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 554/1101] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jun 15 14:32:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:42.542040 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765db5cb-1a29-40d9-aead-757840b08f92 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:42.591728 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 548/1102] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jun 15 14:32:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 15 14:32:42.597442 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fb1d3c-5b96-4348-8a93-466bd0d0e311 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:42.640526 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765db5cb-1a29-40d9-aead-757840b08f92 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:42.675399 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e8c141-4b7b-44fb-988d-e588b5ceffaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:42.676055 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e8c141-4b7b-44fb-988d-e588b5ceffaf None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:42.676370 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e8c141-4b7b-44fb-988d-e588b5ceffaf None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:42.676658 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e8c141-4b7b-44fb-988d-e588b5ceffaf None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:42.677438 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e8c141-4b7b-44fb-988d-e588b5ceffaf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:42.677973 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e8c141-4b7b-44fb-988d-e588b5ceffaf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:42.679163 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e8c141-4b7b-44fb-988d-e588b5ceffaf None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:42.690779 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 555/1103] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jun 15 14:32:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:32:42.695971 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fb1d3c-5b96-4348-8a93-466bd0d0e311 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:42.696679 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e8c141-4b7b-44fb-988d-e588b5ceffaf None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:42.704659 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:42.706061 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:42.729852 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c915e76e-1510-41be-93f6-5b46e3cecb17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:42.730634 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c915e76e-1510-41be-93f6-5b46e3cecb17 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:42.730914 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c915e76e-1510-41be-93f6-5b46e3cecb17 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:42.731189 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c915e76e-1510-41be-93f6-5b46e3cecb17 None admin] PATH_INFO: `/v3/projects/127cea9f376b424586558c0990cbf7ef` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:42.732402 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c915e76e-1510-41be-93f6-5b46e3cecb17 None admin] RBAC: Authorizing `identity:get_project(project_id=127cea9f376b424586558c0990cbf7ef)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:42.733898 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c915e76e-1510-41be-93f6-5b46e3cecb17 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:42.735592 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 549/1104] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jun 15 14:32:42 2022] GET /identity/v3/projects/127cea9f376b424586558c0990cbf7ef => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:42.741632 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c915e76e-1510-41be-93f6-5b46e3cecb17 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:42.797320 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e8c141-4b7b-44fb-988d-e588b5ceffaf None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:42.830481 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd97206-629f-4a51-8d99-9f5a7acc0524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:42.831125 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd97206-629f-4a51-8d99-9f5a7acc0524 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:42.831402 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd97206-629f-4a51-8d99-9f5a7acc0524 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:42.831677 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd97206-629f-4a51-8d99-9f5a7acc0524 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:42.832529 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd97206-629f-4a51-8d99-9f5a7acc0524 None admin] RBAC: Adding query filter params (role.id=76f33d89f76347be9ef2fdac0435173f, scope.project.id=3094ff8966f149bda784da6a51b1358f, user.id=0927c05c340c48e48b18754888aa9c35) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:42.832839 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd97206-629f-4a51-8d99-9f5a7acc0524 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:42.834243 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd97206-629f-4a51-8d99-9f5a7acc0524 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:42.840065 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 556/1105] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jun 15 14:32:42 2022] GET /identity/v3/role_assignments?user.id=0927c05c340c48e48b18754888aa9c35&scope.project.id=3094ff8966f149bda784da6a51b1358f&role.id=76f33d89f76347be9ef2fdac0435173f => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:32:42.842360 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c915e76e-1510-41be-93f6-5b46e3cecb17 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:42.879752 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30af5e7c-18db-472b-83a1-090261645bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:42.880453 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30af5e7c-18db-472b-83a1-090261645bf1 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:42.880740 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30af5e7c-18db-472b-83a1-090261645bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:42.881016 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30af5e7c-18db-472b-83a1-090261645bf1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:42.881852 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30af5e7c-18db-472b-83a1-090261645bf1 None admin] RBAC: Adding query filter params (role.id=2bd1979f53bb4853a50dd5bee6aa0782, scope.project.id=127cea9f376b424586558c0990cbf7ef, user.id=65054510545f44588d9cff7551669fd1) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:42.882166 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30af5e7c-18db-472b-83a1-090261645bf1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:42.883604 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30af5e7c-18db-472b-83a1-090261645bf1 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:42.890349 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 550/1106] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jun 15 14:32:42 2022] GET /identity/v3/role_assignments?user.id=65054510545f44588d9cff7551669fd1&scope.project.id=127cea9f376b424586558c0990cbf7ef&role.id=2bd1979f53bb4853a50dd5bee6aa0782 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:32:45.072112 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f0e77-7999-4e96-b8be-011f114987e5 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:45.072701 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f0e77-7999-4e96-b8be-011f114987e5 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:45.072701 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f0e77-7999-4e96-b8be-011f114987e5 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:45.074514 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 557/1107] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:32:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:32:45.079541 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d175dffb-7999-4c1f-9455-b8dcaa1d53f6 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:45.079945 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d175dffb-7999-4c1f-9455-b8dcaa1d53f6 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:45.080102 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d175dffb-7999-4c1f-9455-b8dcaa1d53f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:45.097117 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-d175dffb-7999-4c1f-9455-b8dcaa1d53f6 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:32:45.098283 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-d175dffb-7999-4c1f-9455-b8dcaa1d53f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:32:45.130869 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:45.163680 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 551/1108] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:32:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:32:45.188579 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619f0e77-7999-4e96-b8be-011f114987e5 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:45.257100 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:45.289094 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619f0e77-7999-4e96-b8be-011f114987e5 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:45.321226 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09aa05a-4900-42f8-82a9-df0164cb0497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:45.322003 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09aa05a-4900-42f8-82a9-df0164cb0497 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:45.322335 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09aa05a-4900-42f8-82a9-df0164cb0497 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:45.322564 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09aa05a-4900-42f8-82a9-df0164cb0497 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:45.324541 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09aa05a-4900-42f8-82a9-df0164cb0497 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:45.325968 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09aa05a-4900-42f8-82a9-df0164cb0497 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:45.327565 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 558/1109] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:32:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:45.333179 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d175dffb-7999-4c1f-9455-b8dcaa1d53f6 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:45.428938 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d175dffb-7999-4c1f-9455-b8dcaa1d53f6 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:45.461913 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733ccdbb-494c-4d3e-90e1-c01f649d38ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:45.462565 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733ccdbb-494c-4d3e-90e1-c01f649d38ba None admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:45.462844 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733ccdbb-494c-4d3e-90e1-c01f649d38ba None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:45.463117 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733ccdbb-494c-4d3e-90e1-c01f649d38ba None admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:45.463898 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733ccdbb-494c-4d3e-90e1-c01f649d38ba None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:45.465336 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733ccdbb-494c-4d3e-90e1-c01f649d38ba None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:45.547302 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 552/1110] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Jun 15 14:32:45 2022] POST /identity/v3/users => generated 286 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:32:47.682922 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597eb0cd-c8bb-4b8c-b8bc-5ef046f83b74 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:47.683636 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597eb0cd-c8bb-4b8c-b8bc-5ef046f83b74 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:47.683742 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597eb0cd-c8bb-4b8c-b8bc-5ef046f83b74 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:47.685211 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 559/1111] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:32:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:32:47.690548 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d38656-764c-4806-b6db-ff74df49533a None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:47.690939 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d38656-764c-4806-b6db-ff74df49533a None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:47.691106 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d38656-764c-4806-b6db-ff74df49533a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:47.708146 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-01d38656-764c-4806-b6db-ff74df49533a None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:32:47.709985 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-01d38656-764c-4806-b6db-ff74df49533a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:32:47.728702 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:47.773879 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 553/1112] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:32:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:32:47.797778 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597eb0cd-c8bb-4b8c-b8bc-5ef046f83b74 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:47.896401 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597eb0cd-c8bb-4b8c-b8bc-5ef046f83b74 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:47.928639 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbaae15-280c-48cc-8d59-8380c3192823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:47.929480 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbaae15-280c-48cc-8d59-8380c3192823 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:47.929762 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbaae15-280c-48cc-8d59-8380c3192823 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:47.930065 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbaae15-280c-48cc-8d59-8380c3192823 None admin] PATH_INFO: `/v3/roles/61e98f694ee741a9894b7e6db6dd11d3` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:47.931514 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbaae15-280c-48cc-8d59-8380c3192823 None admin] RBAC: Authorizing `identity:get_role(role_id=61e98f694ee741a9894b7e6db6dd11d3)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:47.933787 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbaae15-280c-48cc-8d59-8380c3192823 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:47.935144 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 560/1113] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jun 15 14:32:47 2022] GET /identity/v3/roles/61e98f694ee741a9894b7e6db6dd11d3 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:47.940924 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d38656-764c-4806-b6db-ff74df49533a None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:48.036698 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d38656-764c-4806-b6db-ff74df49533a None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:48.069722 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06523a7-957a-4ef2-a696-0bafce34dd3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:48.070577 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06523a7-957a-4ef2-a696-0bafce34dd3d None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:48.070857 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06523a7-957a-4ef2-a696-0bafce34dd3d None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:48.071151 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06523a7-957a-4ef2-a696-0bafce34dd3d None admin] PATH_INFO: `/v3/users/0184d212343d4929a5fef7ff7bd2385e` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:48.083728 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06523a7-957a-4ef2-a696-0bafce34dd3d None admin] RBAC: Authorizing `identity:get_user(user_id=0184d212343d4929a5fef7ff7bd2385e)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:48.085182 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06523a7-957a-4ef2-a696-0bafce34dd3d None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:48.086795 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 554/1114] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jun 15 14:32:47 2022] GET /identity/v3/users/0184d212343d4929a5fef7ff7bd2385e => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:48.092585 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbaae15-280c-48cc-8d59-8380c3192823 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:48.101578 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:48.191446 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbaae15-280c-48cc-8d59-8380c3192823 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:48.223536 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0e2489-687e-4ca5-be2c-014a88f4a39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:48.224196 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0e2489-687e-4ca5-be2c-014a88f4a39d None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:48.224480 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0e2489-687e-4ca5-be2c-014a88f4a39d None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:48.224755 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0e2489-687e-4ca5-be2c-014a88f4a39d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:48.289064 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0e2489-687e-4ca5-be2c-014a88f4a39d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:48.290499 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0e2489-687e-4ca5-be2c-014a88f4a39d None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:48.298365 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:48.384535 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 561/1115] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jun 15 14:32:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:32:48.390893 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06523a7-957a-4ef2-a696-0bafce34dd3d None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:48.407039 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:48.490371 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06523a7-957a-4ef2-a696-0bafce34dd3d None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:48.522701 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104dfc79-0e5d-4b94-a08d-6b242a1eb3b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:48.523787 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104dfc79-0e5d-4b94-a08d-6b242a1eb3b1 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:48.524059 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104dfc79-0e5d-4b94-a08d-6b242a1eb3b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:48.524365 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104dfc79-0e5d-4b94-a08d-6b242a1eb3b1 None admin] PATH_INFO: `/v3/projects/127cea9f376b424586558c0990cbf7ef` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:48.525516 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104dfc79-0e5d-4b94-a08d-6b242a1eb3b1 None admin] RBAC: Authorizing `identity:get_project(project_id=127cea9f376b424586558c0990cbf7ef)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:48.527554 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104dfc79-0e5d-4b94-a08d-6b242a1eb3b1 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:48.529245 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 555/1116] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jun 15 14:32:48 2022] GET /identity/v3/projects/127cea9f376b424586558c0990cbf7ef => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:48.535282 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0e2489-687e-4ca5-be2c-014a88f4a39d None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:48.630945 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0e2489-687e-4ca5-be2c-014a88f4a39d None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:48.638386 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:48.664249 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b29ea1b-e294-42bf-adcc-9ecbaa554866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:48.664898 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b29ea1b-e294-42bf-adcc-9ecbaa554866 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:48.665178 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b29ea1b-e294-42bf-adcc-9ecbaa554866 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:48.665451 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b29ea1b-e294-42bf-adcc-9ecbaa554866 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:48.666291 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b29ea1b-e294-42bf-adcc-9ecbaa554866 None admin] RBAC: Adding query filter params (role.id=61e98f694ee741a9894b7e6db6dd11d3, scope.project.id=127cea9f376b424586558c0990cbf7ef, user.id=0184d212343d4929a5fef7ff7bd2385e) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:48.666784 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b29ea1b-e294-42bf-adcc-9ecbaa554866 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:48.668289 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b29ea1b-e294-42bf-adcc-9ecbaa554866 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:48.673591 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 562/1117] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jun 15 14:32:48 2022] GET /identity/v3/role_assignments?user.id=0184d212343d4929a5fef7ff7bd2385e&scope.project.id=127cea9f376b424586558c0990cbf7ef&role.id=61e98f694ee741a9894b7e6db6dd11d3 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:50.737415 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f394f8f3-01cf-428a-b95e-0887506cbaa4 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:50.737970 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f394f8f3-01cf-428a-b95e-0887506cbaa4 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:50.737970 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f394f8f3-01cf-428a-b95e-0887506cbaa4 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:50.739301 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 556/1118] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:32:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:32:50.744367 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9795b4eb-bee8-4f8b-978e-74b920287bec None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:50.744779 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9795b4eb-bee8-4f8b-978e-74b920287bec None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:50.745248 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9795b4eb-bee8-4f8b-978e-74b920287bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:50.762168 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-9795b4eb-bee8-4f8b-978e-74b920287bec None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:32:50.763336 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-9795b4eb-bee8-4f8b-978e-74b920287bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:32:50.829340 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 563/1119] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:32:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:32:50.852450 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f394f8f3-01cf-428a-b95e-0887506cbaa4 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:50.934031 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:50.951896 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f394f8f3-01cf-428a-b95e-0887506cbaa4 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:50.984545 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a4675a-d7b2-4cbd-b520-e586fb5d217b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:50.985381 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a4675a-d7b2-4cbd-b520-e586fb5d217b None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:50.985658 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a4675a-d7b2-4cbd-b520-e586fb5d217b None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:50.985927 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a4675a-d7b2-4cbd-b520-e586fb5d217b None admin] PATH_INFO: `/v3/roles/61e98f694ee741a9894b7e6db6dd11d3` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:50.987352 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a4675a-d7b2-4cbd-b520-e586fb5d217b None admin] RBAC: Authorizing `identity:get_role(role_id=61e98f694ee741a9894b7e6db6dd11d3)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:50.988728 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a4675a-d7b2-4cbd-b520-e586fb5d217b None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:50.990074 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 557/1120] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jun 15 14:32:50 2022] GET /identity/v3/roles/61e98f694ee741a9894b7e6db6dd11d3 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:50.996830 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9795b4eb-bee8-4f8b-978e-74b920287bec None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:51.087565 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:51.090361 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9795b4eb-bee8-4f8b-978e-74b920287bec None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:51.124265 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e2d80d-26fb-4bda-bfbc-d44d0587e971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:51.125133 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e2d80d-26fb-4bda-bfbc-d44d0587e971 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:51.125422 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e2d80d-26fb-4bda-bfbc-d44d0587e971 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:51.125703 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e2d80d-26fb-4bda-bfbc-d44d0587e971 None admin] PATH_INFO: `/v3/users/0184d212343d4929a5fef7ff7bd2385e` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:51.126888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e2d80d-26fb-4bda-bfbc-d44d0587e971 None admin] RBAC: Authorizing `identity:get_user(user_id=0184d212343d4929a5fef7ff7bd2385e)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:51.128372 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e2d80d-26fb-4bda-bfbc-d44d0587e971 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:51.130010 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 564/1121] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jun 15 14:32:50 2022] GET /identity/v3/users/0184d212343d4929a5fef7ff7bd2385e => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:32:51.135837 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a4675a-d7b2-4cbd-b520-e586fb5d217b None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:51.234038 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a4675a-d7b2-4cbd-b520-e586fb5d217b None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:51.266807 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9a1101-fa57-46fc-af84-92c87dc87615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:51.267708 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9a1101-fa57-46fc-af84-92c87dc87615 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:51.268022 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9a1101-fa57-46fc-af84-92c87dc87615 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:51.268355 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9a1101-fa57-46fc-af84-92c87dc87615 None admin] PATH_INFO: `/v3/projects/127cea9f376b424586558c0990cbf7ef` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:51.269570 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9a1101-fa57-46fc-af84-92c87dc87615 None admin] RBAC: Authorizing `identity:get_project(project_id=127cea9f376b424586558c0990cbf7ef)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:51.271112 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9a1101-fa57-46fc-af84-92c87dc87615 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:51.272806 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 558/1122] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jun 15 14:32:51 2022] GET /identity/v3/projects/127cea9f376b424586558c0990cbf7ef => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:51.279334 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e2d80d-26fb-4bda-bfbc-d44d0587e971 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:51.374980 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e2d80d-26fb-4bda-bfbc-d44d0587e971 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:51.407665 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c888be-6c9c-496a-85c0-06fe38233a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:51.408629 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c888be-6c9c-496a-85c0-06fe38233a2e None admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:51.408909 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c888be-6c9c-496a-85c0-06fe38233a2e None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:51.409188 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c888be-6c9c-496a-85c0-06fe38233a2e None admin] PATH_INFO: `/v3/projects/127cea9f376b424586558c0990cbf7ef/users/0184d212343d4929a5fef7ff7bd2385e/roles/61e98f694ee741a9894b7e6db6dd11d3` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:51.411356 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c888be-6c9c-496a-85c0-06fe38233a2e None admin] RBAC: Authorizing `identity:create_grant(project_id=127cea9f376b424586558c0990cbf7ef, user_id=0184d212343d4929a5fef7ff7bd2385e, role_id=61e98f694ee741a9894b7e6db6dd11d3)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:51.412791 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c888be-6c9c-496a-85c0-06fe38233a2e None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:51.477799 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 565/1123] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Jun 15 14:32:51 2022] PUT /identity/v3/projects/127cea9f376b424586558c0990cbf7ef/users/0184d212343d4929a5fef7ff7bd2385e/roles/61e98f694ee741a9894b7e6db6dd11d3 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:32:53.574806 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a184c60-b38b-4172-b948-219a621af995 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:53.575304 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a184c60-b38b-4172-b948-219a621af995 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:53.575408 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a184c60-b38b-4172-b948-219a621af995 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:53.576772 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 559/1124] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:32:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:32:53.581943 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68c15ec-7682-4e91-a9c2-b29bfd6f21d3 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:53.582345 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68c15ec-7682-4e91-a9c2-b29bfd6f21d3 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:53.582494 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68c15ec-7682-4e91-a9c2-b29bfd6f21d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:53.600421 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-c68c15ec-7682-4e91-a9c2-b29bfd6f21d3 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:32:53.601636 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c68c15ec-7682-4e91-a9c2-b29bfd6f21d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:32:53.624450 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:53.668237 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 566/1125] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:32:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:32:53.691740 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a184c60-b38b-4172-b948-219a621af995 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:53.710120 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:53.792910 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a184c60-b38b-4172-b948-219a621af995 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:53.825555 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d613e00-f6c9-434a-ae5f-07400a93b6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:53.826317 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d613e00-f6c9-434a-ae5f-07400a93b6bf None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:53.826593 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d613e00-f6c9-434a-ae5f-07400a93b6bf None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:53.826863 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d613e00-f6c9-434a-ae5f-07400a93b6bf None admin] PATH_INFO: `/v3/roles/61e98f694ee741a9894b7e6db6dd11d3` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:53.828314 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d613e00-f6c9-434a-ae5f-07400a93b6bf None admin] RBAC: Authorizing `identity:get_role(role_id=61e98f694ee741a9894b7e6db6dd11d3)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:53.829672 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d613e00-f6c9-434a-ae5f-07400a93b6bf None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:53.831008 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 560/1126] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jun 15 14:32:53 2022] GET /identity/v3/roles/61e98f694ee741a9894b7e6db6dd11d3 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:53.836964 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68c15ec-7682-4e91-a9c2-b29bfd6f21d3 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:53.934944 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68c15ec-7682-4e91-a9c2-b29bfd6f21d3 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:53.968457 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53fdfa0-3c26-46a2-b22c-151c4452bad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:53.969320 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53fdfa0-3c26-46a2-b22c-151c4452bad0 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:53.969602 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53fdfa0-3c26-46a2-b22c-151c4452bad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:53.969877 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53fdfa0-3c26-46a2-b22c-151c4452bad0 None admin] PATH_INFO: `/v3/users/0184d212343d4929a5fef7ff7bd2385e` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:53.971057 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53fdfa0-3c26-46a2-b22c-151c4452bad0 None admin] RBAC: Authorizing `identity:get_user(user_id=0184d212343d4929a5fef7ff7bd2385e)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:53.972517 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53fdfa0-3c26-46a2-b22c-151c4452bad0 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:53.974150 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 567/1127] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jun 15 14:32:53 2022] GET /identity/v3/users/0184d212343d4929a5fef7ff7bd2385e => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:53.979968 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d613e00-f6c9-434a-ae5f-07400a93b6bf None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:54.061468 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:54.076071 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d613e00-f6c9-434a-ae5f-07400a93b6bf None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:54.108770 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0c1892-88e1-40a7-8af3-e8e21bba5499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:54.109399 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0c1892-88e1-40a7-8af3-e8e21bba5499 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:54.109676 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0c1892-88e1-40a7-8af3-e8e21bba5499 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:54.109947 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0c1892-88e1-40a7-8af3-e8e21bba5499 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:54.175265 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0c1892-88e1-40a7-8af3-e8e21bba5499 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:54.176749 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0c1892-88e1-40a7-8af3-e8e21bba5499 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:54.259796 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:54.281435 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 561/1128] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jun 15 14:32:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:32:54.282344 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53fdfa0-3c26-46a2-b22c-151c4452bad0 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:54.311415 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:54.385475 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53fdfa0-3c26-46a2-b22c-151c4452bad0 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:54.420299 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5bb5f3-e32a-45a2-9021-2b63632ff7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:54.421163 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5bb5f3-e32a-45a2-9021-2b63632ff7f8 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:54.421450 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5bb5f3-e32a-45a2-9021-2b63632ff7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:54.421730 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5bb5f3-e32a-45a2-9021-2b63632ff7f8 None admin] PATH_INFO: `/v3/projects/127cea9f376b424586558c0990cbf7ef` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:54.422930 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5bb5f3-e32a-45a2-9021-2b63632ff7f8 None admin] RBAC: Authorizing `identity:get_project(project_id=127cea9f376b424586558c0990cbf7ef)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:54.424397 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5bb5f3-e32a-45a2-9021-2b63632ff7f8 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:54.426260 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 568/1129] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jun 15 14:32:54 2022] GET /identity/v3/projects/127cea9f376b424586558c0990cbf7ef => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:54.432504 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0c1892-88e1-40a7-8af3-e8e21bba5499 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:54.532749 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0c1892-88e1-40a7-8af3-e8e21bba5499 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:54.565280 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c6a62d-26d1-4c7a-83da-830b37002ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:54.565928 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6a62d-26d1-4c7a-83da-830b37002ecb None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:54.566213 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6a62d-26d1-4c7a-83da-830b37002ecb None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:54.566491 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6a62d-26d1-4c7a-83da-830b37002ecb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:54.567315 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c6a62d-26d1-4c7a-83da-830b37002ecb None admin] RBAC: Adding query filter params (role.id=61e98f694ee741a9894b7e6db6dd11d3, scope.project.id=127cea9f376b424586558c0990cbf7ef, user.id=0184d212343d4929a5fef7ff7bd2385e) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:54.567627 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c6a62d-26d1-4c7a-83da-830b37002ecb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:54.569078 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c6a62d-26d1-4c7a-83da-830b37002ecb None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:54.575453 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 562/1130] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jun 15 14:32:54 2022] GET /identity/v3/role_assignments?user.id=0184d212343d4929a5fef7ff7bd2385e&scope.project.id=127cea9f376b424586558c0990cbf7ef&role.id=61e98f694ee741a9894b7e6db6dd11d3 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:56.712247 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b9f86b-3368-4679-a636-d498d1a86c23 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:56.712782 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b9f86b-3368-4679-a636-d498d1a86c23 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:56.712782 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b9f86b-3368-4679-a636-d498d1a86c23 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:56.714215 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 569/1131] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:32:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:32:56.719326 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3748b8e-2397-46d9-9956-060351873d43 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:56.719751 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3748b8e-2397-46d9-9956-060351873d43 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:56.719878 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3748b8e-2397-46d9-9956-060351873d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:56.736638 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-b3748b8e-2397-46d9-9956-060351873d43 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:32:56.737806 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-b3748b8e-2397-46d9-9956-060351873d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:32:56.782124 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:56.802356 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 563/1132] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:32:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:32:56.826453 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b9f86b-3368-4679-a636-d498d1a86c23 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:56.917725 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:56.925119 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b9f86b-3368-4679-a636-d498d1a86c23 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:56.958607 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9fc955-946a-450a-b130-6af1e9ccc238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:56.959381 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9fc955-946a-450a-b130-6af1e9ccc238 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:56.959662 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9fc955-946a-450a-b130-6af1e9ccc238 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:56.959937 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9fc955-946a-450a-b130-6af1e9ccc238 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:56.961151 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9fc955-946a-450a-b130-6af1e9ccc238 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:56.962563 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9fc955-946a-450a-b130-6af1e9ccc238 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:56.964625 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 570/1133] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:32:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:56.970131 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3748b8e-2397-46d9-9956-060351873d43 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:57.066461 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3748b8e-2397-46d9-9956-060351873d43 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:57.098715 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abba93ea-ed30-4170-bd62-08aa25f380f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:57.099305 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abba93ea-ed30-4170-bd62-08aa25f380f5 None admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:57.099590 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abba93ea-ed30-4170-bd62-08aa25f380f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:57.099869 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abba93ea-ed30-4170-bd62-08aa25f380f5 None admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:57.100775 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abba93ea-ed30-4170-bd62-08aa25f380f5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:57.102196 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abba93ea-ed30-4170-bd62-08aa25f380f5 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:57.227467 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 564/1134] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Jun 15 14:32:56 2022] POST /identity/v3/projects => generated 311 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:32:59.357541 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a3e4a2-1c42-4c90-99eb-39fdf70a2074 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:59.358073 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a3e4a2-1c42-4c90-99eb-39fdf70a2074 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:59.358073 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a3e4a2-1c42-4c90-99eb-39fdf70a2074 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:59.359444 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 571/1135] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:32:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:32:59.364632 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c26a76-9bba-41f1-b381-09b4ec524296 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:59.365002 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c26a76-9bba-41f1-b381-09b4ec524296 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:59.365216 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c26a76-9bba-41f1-b381-09b4ec524296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:59.381863 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-30c26a76-9bba-41f1-b381-09b4ec524296 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:32:59.383029 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-30c26a76-9bba-41f1-b381-09b4ec524296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:32:59.408414 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:59.449824 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 565/1136] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:32:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:32:59.474109 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a3e4a2-1c42-4c90-99eb-39fdf70a2074 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:59.575589 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a3e4a2-1c42-4c90-99eb-39fdf70a2074 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:59.608157 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed82ef8-30da-444d-957e-25c6d9c9462a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:59.608892 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed82ef8-30da-444d-957e-25c6d9c9462a None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:59.609173 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed82ef8-30da-444d-957e-25c6d9c9462a None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:59.609449 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed82ef8-30da-444d-957e-25c6d9c9462a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:59.610600 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed82ef8-30da-444d-957e-25c6d9c9462a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:59.612022 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed82ef8-30da-444d-957e-25c6d9c9462a None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:59.613657 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 572/1137] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:32:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:59.619331 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c26a76-9bba-41f1-b381-09b4ec524296 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:59.715676 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c26a76-9bba-41f1-b381-09b4ec524296 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:59.749424 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8748f37-9843-4d79-8b5b-01d0c842d8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:59.750075 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8748f37-9843-4d79-8b5b-01d0c842d8ca None admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:59.750354 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8748f37-9843-4d79-8b5b-01d0c842d8ca None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:59.750625 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8748f37-9843-4d79-8b5b-01d0c842d8ca None admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:59.751332 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8748f37-9843-4d79-8b5b-01d0c842d8ca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:59.752776 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8748f37-9843-4d79-8b5b-01d0c842d8ca None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:59.826337 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 566/1138] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Jun 15 14:32:59 2022] POST /identity/v3/users => generated 286 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:33:01.964348 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974122d6-5f90-4e4f-810f-b774da07d2d1 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:01.964909 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974122d6-5f90-4e4f-810f-b774da07d2d1 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:01.964909 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974122d6-5f90-4e4f-810f-b774da07d2d1 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:01.966287 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 573/1139] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:33:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:33:01.971735 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7905fe8-4453-4c6a-ace1-32b89cb794f2 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:01.972173 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7905fe8-4453-4c6a-ace1-32b89cb794f2 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:01.972331 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7905fe8-4453-4c6a-ace1-32b89cb794f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:01.989405 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-d7905fe8-4453-4c6a-ace1-32b89cb794f2 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:33:01.990688 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-d7905fe8-4453-4c6a-ace1-32b89cb794f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:33:02.004387 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:02.058345 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 567/1140] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:33:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:33:02.081309 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974122d6-5f90-4e4f-810f-b774da07d2d1 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:02.111216 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:02.180159 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974122d6-5f90-4e4f-810f-b774da07d2d1 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:02.212507 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec7312f-0d32-4d6f-8f83-0b2f59c3fe29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:02.213278 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec7312f-0d32-4d6f-8f83-0b2f59c3fe29 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:02.213557 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec7312f-0d32-4d6f-8f83-0b2f59c3fe29 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:02.213853 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec7312f-0d32-4d6f-8f83-0b2f59c3fe29 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:02.223706 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec7312f-0d32-4d6f-8f83-0b2f59c3fe29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:33:02.231883 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-5ec7312f-0d32-4d6f-8f83-0b2f59c3fe29 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:33:02.231883 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:33:02.231883 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:33:02.231883 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:33:02.231883 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:33:02.231883 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:33:02.231883 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:33:02.231883 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:33:02.231883 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:33:02.231883 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:33:02.231883 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:33:02.231883 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:33:02.231883 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:33:02.231883 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:33:02.231883 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:33:02.231883 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:33:02.233153 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:33:02.233153 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:33:02.233153 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:33:02.233153 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:33:02.233153 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:33:02.233153 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:33:02.233153 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:33:02.233153 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:33:02.233153 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:33:02.233153 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:33:02.233153 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:33:02.233153 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:33:02.233153 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:33:02.233153 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:33:02.233153 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:33:02.233153 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:33:02.234423 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:33:02.234423 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:33:02.234423 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:33:02.234423 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:33:02.234423 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:33:02.234423 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:33:02.234423 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:33:02.234423 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:33:02.234423 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:33:02.234423 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:33:02.234423 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:33:02.234423 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:33:02.234423 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 574/1141] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jun 15 14:33:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:33:02.238990 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7905fe8-4453-4c6a-ace1-32b89cb794f2 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:02.335258 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7905fe8-4453-4c6a-ace1-32b89cb794f2 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:02.367582 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e2cd32-0de6-4132-b770-3ccad3f9b132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:02.368257 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e2cd32-0de6-4132-b770-3ccad3f9b132 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:02.368538 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e2cd32-0de6-4132-b770-3ccad3f9b132 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:02.368813 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e2cd32-0de6-4132-b770-3ccad3f9b132 None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:02.369540 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e2cd32-0de6-4132-b770-3ccad3f9b132 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:33:02.369843 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e2cd32-0de6-4132-b770-3ccad3f9b132 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:02.371853 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e2cd32-0de6-4132-b770-3ccad3f9b132 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:02.379531 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 568/1142] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jun 15 14:33:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:33:02.385240 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec7312f-0d32-4d6f-8f83-0b2f59c3fe29 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:02.473575 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:02.489133 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec7312f-0d32-4d6f-8f83-0b2f59c3fe29 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:02.516528 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a41913b-1df9-44fc-850d-dcc98b7dbc9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:02.517380 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a41913b-1df9-44fc-850d-dcc98b7dbc9f None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:02.517662 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a41913b-1df9-44fc-850d-dcc98b7dbc9f None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:02.517941 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a41913b-1df9-44fc-850d-dcc98b7dbc9f None admin] PATH_INFO: `/v3/users/a1fe605d4b944c04a09bb8261f523c64` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:02.531613 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a41913b-1df9-44fc-850d-dcc98b7dbc9f None admin] RBAC: Authorizing `identity:get_user(user_id=a1fe605d4b944c04a09bb8261f523c64)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:02.533065 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a41913b-1df9-44fc-850d-dcc98b7dbc9f None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:02.534702 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 575/1143] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jun 15 14:33:02 2022] GET /identity/v3/users/a1fe605d4b944c04a09bb8261f523c64 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:33:02.540645 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e2cd32-0de6-4132-b770-3ccad3f9b132 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:02.554920 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:02.642412 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e2cd32-0de6-4132-b770-3ccad3f9b132 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:02.674006 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be07436-c83b-4335-9a66-05c80864f51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:02.674634 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be07436-c83b-4335-9a66-05c80864f51f None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:02.674912 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be07436-c83b-4335-9a66-05c80864f51f None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:02.675183 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be07436-c83b-4335-9a66-05c80864f51f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:02.739552 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be07436-c83b-4335-9a66-05c80864f51f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:02.740940 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be07436-c83b-4335-9a66-05c80864f51f None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:02.751935 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:02.834583 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 569/1144] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jun 15 14:33:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:33:02.841127 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a41913b-1df9-44fc-850d-dcc98b7dbc9f None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:02.936581 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a41913b-1df9-44fc-850d-dcc98b7dbc9f None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:02.969528 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9008bf7-9ef7-462e-9a45-d5df0af79789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:02.970374 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9008bf7-9ef7-462e-9a45-d5df0af79789 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:02.970657 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9008bf7-9ef7-462e-9a45-d5df0af79789 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:02.970935 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9008bf7-9ef7-462e-9a45-d5df0af79789 None admin] PATH_INFO: `/v3/projects/f3b90869933b4b20a5ca1fee6006b44c` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:02.972124 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9008bf7-9ef7-462e-9a45-d5df0af79789 None admin] RBAC: Authorizing `identity:get_project(project_id=f3b90869933b4b20a5ca1fee6006b44c)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:02.973534 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9008bf7-9ef7-462e-9a45-d5df0af79789 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:02.975196 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 576/1145] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jun 15 14:33:02 2022] GET /identity/v3/projects/f3b90869933b4b20a5ca1fee6006b44c => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:33:02.981151 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be07436-c83b-4335-9a66-05c80864f51f None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:03.075222 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be07436-c83b-4335-9a66-05c80864f51f None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:03.085571 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:03.108302 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3c0d7b-7f4b-4780-9df0-59cedcd8598a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:03.108972 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3c0d7b-7f4b-4780-9df0-59cedcd8598a None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:03.109256 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3c0d7b-7f4b-4780-9df0-59cedcd8598a None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:03.109533 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3c0d7b-7f4b-4780-9df0-59cedcd8598a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:03.110349 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3c0d7b-7f4b-4780-9df0-59cedcd8598a None admin] RBAC: Adding query filter params (role.id=2bd1979f53bb4853a50dd5bee6aa0782, scope.project.id=f3b90869933b4b20a5ca1fee6006b44c, user.id=a1fe605d4b944c04a09bb8261f523c64) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:33:03.110663 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3c0d7b-7f4b-4780-9df0-59cedcd8598a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:03.112126 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3c0d7b-7f4b-4780-9df0-59cedcd8598a None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:03.117805 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 570/1146] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jun 15 14:33:02 2022] GET /identity/v3/role_assignments?user.id=a1fe605d4b944c04a09bb8261f523c64&scope.project.id=f3b90869933b4b20a5ca1fee6006b44c&role.id=2bd1979f53bb4853a50dd5bee6aa0782 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:33:05.181826 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ef7f2d-7139-4268-ad22-c9dc547e634d None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:05.182386 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ef7f2d-7139-4268-ad22-c9dc547e634d None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:05.182386 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ef7f2d-7139-4268-ad22-c9dc547e634d None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:05.184036 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 577/1147] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:33:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:33:05.189170 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95d48f-fbb9-4941-85d3-fc6df968ec78 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:05.189582 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95d48f-fbb9-4941-85d3-fc6df968ec78 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:05.189722 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95d48f-fbb9-4941-85d3-fc6df968ec78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:05.206422 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-9e95d48f-fbb9-4941-85d3-fc6df968ec78 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:33:05.207579 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-9e95d48f-fbb9-4941-85d3-fc6df968ec78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:33:05.271001 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 571/1148] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:33:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:33:05.293727 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ef7f2d-7139-4268-ad22-c9dc547e634d None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:05.311392 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:05.391731 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ef7f2d-7139-4268-ad22-c9dc547e634d None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:05.423861 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06794f8-4432-4a9f-86e0-45ab55837d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:05.424660 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06794f8-4432-4a9f-86e0-45ab55837d10 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:05.424944 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06794f8-4432-4a9f-86e0-45ab55837d10 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:05.425218 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06794f8-4432-4a9f-86e0-45ab55837d10 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:05.435599 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06794f8-4432-4a9f-86e0-45ab55837d10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:33:05.443089 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-a06794f8-4432-4a9f-86e0-45ab55837d10 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:33:05.443089 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:33:05.443089 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:33:05.443089 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:33:05.443089 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:33:05.443089 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:33:05.443089 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:33:05.443089 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:33:05.443089 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:33:05.443089 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:33:05.443089 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:33:05.443089 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:33:05.443089 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:33:05.443089 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:33:05.443089 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:33:05.443089 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:33:05.444329 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:33:05.444329 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:33:05.444329 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:33:05.444329 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:33:05.444329 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:33:05.444329 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:33:05.444329 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:33:05.444329 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:33:05.444329 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:33:05.444329 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:33:05.444329 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:33:05.444329 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:33:05.444329 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:33:05.444329 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:33:05.444329 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:33:05.444329 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:33:05.445517 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:33:05.445517 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:33:05.445517 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:33:05.445517 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:33:05.445517 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:33:05.445517 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:33:05.445517 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:33:05.445517 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:33:05.445517 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:33:05.445517 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:33:05.445517 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:33:05.445517 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:33:05.445517 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 578/1149] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jun 15 14:33:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:33:05.450919 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e95d48f-fbb9-4941-85d3-fc6df968ec78 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:05.546579 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e95d48f-fbb9-4941-85d3-fc6df968ec78 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:05.548867 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:05.580946 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3dd3cf-ec59-4242-9f0e-1032ccaf433f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:05.581833 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3dd3cf-ec59-4242-9f0e-1032ccaf433f None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:05.582411 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3dd3cf-ec59-4242-9f0e-1032ccaf433f None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:05.582694 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3dd3cf-ec59-4242-9f0e-1032ccaf433f None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:05.583442 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3dd3cf-ec59-4242-9f0e-1032ccaf433f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:33:05.583753 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3dd3cf-ec59-4242-9f0e-1032ccaf433f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:05.585158 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3dd3cf-ec59-4242-9f0e-1032ccaf433f None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:05.593238 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 572/1150] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jun 15 14:33:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:33:05.598961 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06794f8-4432-4a9f-86e0-45ab55837d10 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:05.673500 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:05.695749 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06794f8-4432-4a9f-86e0-45ab55837d10 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:05.727634 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bca6916-3bb6-43ac-a321-6a13f3d88e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:05.728521 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca6916-3bb6-43ac-a321-6a13f3d88e62 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:05.728799 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca6916-3bb6-43ac-a321-6a13f3d88e62 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:05.729067 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca6916-3bb6-43ac-a321-6a13f3d88e62 None admin] PATH_INFO: `/v3/users/a1fe605d4b944c04a09bb8261f523c64` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:05.730815 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bca6916-3bb6-43ac-a321-6a13f3d88e62 None admin] RBAC: Authorizing `identity:get_user(user_id=a1fe605d4b944c04a09bb8261f523c64)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:05.732248 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bca6916-3bb6-43ac-a321-6a13f3d88e62 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:05.733855 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 579/1151] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jun 15 14:33:05 2022] GET /identity/v3/users/a1fe605d4b944c04a09bb8261f523c64 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:33:05.739632 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3dd3cf-ec59-4242-9f0e-1032ccaf433f None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:05.836791 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3dd3cf-ec59-4242-9f0e-1032ccaf433f None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:05.875304 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa2a369-4d83-4c13-a097-2ef377d041d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:05.876169 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa2a369-4d83-4c13-a097-2ef377d041d2 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:05.876448 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa2a369-4d83-4c13-a097-2ef377d041d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:05.876719 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa2a369-4d83-4c13-a097-2ef377d041d2 None admin] PATH_INFO: `/v3/projects/f3b90869933b4b20a5ca1fee6006b44c` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:05.877865 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa2a369-4d83-4c13-a097-2ef377d041d2 None admin] RBAC: Authorizing `identity:get_project(project_id=f3b90869933b4b20a5ca1fee6006b44c)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:05.879275 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa2a369-4d83-4c13-a097-2ef377d041d2 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:05.880962 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 573/1152] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jun 15 14:33:05 2022] GET /identity/v3/projects/f3b90869933b4b20a5ca1fee6006b44c => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:33:05.886998 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bca6916-3bb6-43ac-a321-6a13f3d88e62 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:05.984609 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bca6916-3bb6-43ac-a321-6a13f3d88e62 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:06.017443 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33177cc0-50e8-462d-8159-ff087448eb4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:06.018321 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33177cc0-50e8-462d-8159-ff087448eb4c None admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:06.018605 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33177cc0-50e8-462d-8159-ff087448eb4c None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:06.018887 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33177cc0-50e8-462d-8159-ff087448eb4c None admin] PATH_INFO: `/v3/projects/f3b90869933b4b20a5ca1fee6006b44c/users/a1fe605d4b944c04a09bb8261f523c64/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:06.020882 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33177cc0-50e8-462d-8159-ff087448eb4c None admin] RBAC: Authorizing `identity:create_grant(project_id=f3b90869933b4b20a5ca1fee6006b44c, user_id=a1fe605d4b944c04a09bb8261f523c64, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:06.022293 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33177cc0-50e8-462d-8159-ff087448eb4c None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:06.100631 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 580/1153] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Jun 15 14:33:05 2022] PUT /identity/v3/projects/f3b90869933b4b20a5ca1fee6006b44c/users/a1fe605d4b944c04a09bb8261f523c64/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:33:08.191023 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e652f78-9e2e-47a9-a01e-8a7a31cd5125 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:08.191550 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e652f78-9e2e-47a9-a01e-8a7a31cd5125 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:08.191550 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e652f78-9e2e-47a9-a01e-8a7a31cd5125 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:08.192952 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 574/1154] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:33:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:33:08.197957 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362bf82-b5e5-48e5-87bf-4c4200f0338a None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:08.198374 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362bf82-b5e5-48e5-87bf-4c4200f0338a None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:08.198512 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362bf82-b5e5-48e5-87bf-4c4200f0338a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:08.204412 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:08.214924 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-3362bf82-b5e5-48e5-87bf-4c4200f0338a None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:33:08.216167 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-3362bf82-b5e5-48e5-87bf-4c4200f0338a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:33:08.282485 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 581/1155] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:33:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:33:08.306189 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e652f78-9e2e-47a9-a01e-8a7a31cd5125 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:08.340610 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:08.405756 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e652f78-9e2e-47a9-a01e-8a7a31cd5125 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:08.438212 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc133cd-4ed6-4586-ac08-992010c6a8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:08.438991 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc133cd-4ed6-4586-ac08-992010c6a8af None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:08.439278 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc133cd-4ed6-4586-ac08-992010c6a8af None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:08.439558 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc133cd-4ed6-4586-ac08-992010c6a8af None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:08.449945 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc133cd-4ed6-4586-ac08-992010c6a8af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:33:08.457502 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-dcc133cd-4ed6-4586-ac08-992010c6a8af None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:33:08.457502 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:33:08.457502 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:33:08.457502 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:33:08.457502 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:33:08.457502 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:33:08.457502 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:33:08.457502 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:33:08.457502 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:33:08.457502 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:33:08.457502 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:33:08.457502 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:33:08.457502 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:33:08.457502 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:33:08.457502 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:33:08.457502 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:33:08.458609 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:33:08.458609 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:33:08.458609 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:33:08.458609 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:33:08.458609 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:33:08.458609 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:33:08.458609 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:33:08.458609 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:33:08.458609 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:33:08.458609 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:33:08.458609 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:33:08.458609 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:33:08.458609 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:33:08.458609 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:33:08.458609 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:33:08.458609 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:33:08.459853 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:33:08.459853 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:33:08.459853 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:33:08.459853 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:33:08.459853 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:33:08.459853 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:33:08.459853 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:33:08.459853 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:33:08.459853 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:33:08.459853 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:33:08.459853 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:33:08.459853 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:33:08.459853 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 575/1156] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jun 15 14:33:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:33:08.465274 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3362bf82-b5e5-48e5-87bf-4c4200f0338a None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:08.560142 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3362bf82-b5e5-48e5-87bf-4c4200f0338a None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:08.594557 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28400d3-b5e1-4fca-8d0e-c69aff982289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:08.595215 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28400d3-b5e1-4fca-8d0e-c69aff982289 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:08.595600 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28400d3-b5e1-4fca-8d0e-c69aff982289 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:08.595964 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28400d3-b5e1-4fca-8d0e-c69aff982289 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:08.596750 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28400d3-b5e1-4fca-8d0e-c69aff982289 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:33:08.597061 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28400d3-b5e1-4fca-8d0e-c69aff982289 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:08.598413 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28400d3-b5e1-4fca-8d0e-c69aff982289 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:08.599735 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:08.607325 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 582/1157] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jun 15 14:33:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:33:08.613039 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc133cd-4ed6-4586-ac08-992010c6a8af None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:08.708532 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:08.715491 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc133cd-4ed6-4586-ac08-992010c6a8af None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:08.748512 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d02df59-6b82-4138-a4c4-de8c5b18e681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:08.749356 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d02df59-6b82-4138-a4c4-de8c5b18e681 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:08.749637 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d02df59-6b82-4138-a4c4-de8c5b18e681 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:08.749908 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d02df59-6b82-4138-a4c4-de8c5b18e681 None admin] PATH_INFO: `/v3/users/a1fe605d4b944c04a09bb8261f523c64` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:08.751065 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d02df59-6b82-4138-a4c4-de8c5b18e681 None admin] RBAC: Authorizing `identity:get_user(user_id=a1fe605d4b944c04a09bb8261f523c64)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:08.752504 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d02df59-6b82-4138-a4c4-de8c5b18e681 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:08.754091 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 576/1158] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jun 15 14:33:08 2022] GET /identity/v3/users/a1fe605d4b944c04a09bb8261f523c64 => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:33:08.759813 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28400d3-b5e1-4fca-8d0e-c69aff982289 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:08.855956 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28400d3-b5e1-4fca-8d0e-c69aff982289 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:08.887568 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fcb11f-aef7-4f72-839a-32f3e6694b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:08.888230 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fcb11f-aef7-4f72-839a-32f3e6694b63 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:08.888510 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fcb11f-aef7-4f72-839a-32f3e6694b63 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:08.888783 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fcb11f-aef7-4f72-839a-32f3e6694b63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:08.946858 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:08.953765 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fcb11f-aef7-4f72-839a-32f3e6694b63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:08.955237 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fcb11f-aef7-4f72-839a-32f3e6694b63 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:09.050391 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 583/1159] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jun 15 14:33:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:33:09.056570 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d02df59-6b82-4138-a4c4-de8c5b18e681 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:09.154152 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d02df59-6b82-4138-a4c4-de8c5b18e681 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:09.186836 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f89329-2e06-4397-afd1-4ffbc1c24998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:09.187672 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f89329-2e06-4397-afd1-4ffbc1c24998 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:09.187954 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f89329-2e06-4397-afd1-4ffbc1c24998 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:09.188258 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f89329-2e06-4397-afd1-4ffbc1c24998 None admin] PATH_INFO: `/v3/projects/f3b90869933b4b20a5ca1fee6006b44c` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:09.189421 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f89329-2e06-4397-afd1-4ffbc1c24998 None admin] RBAC: Authorizing `identity:get_project(project_id=f3b90869933b4b20a5ca1fee6006b44c)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:09.190833 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f89329-2e06-4397-afd1-4ffbc1c24998 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:09.192536 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 577/1160] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jun 15 14:33:09 2022] GET /identity/v3/projects/f3b90869933b4b20a5ca1fee6006b44c => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:33:09.198613 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fcb11f-aef7-4f72-839a-32f3e6694b63 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:09.287976 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:09.294387 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fcb11f-aef7-4f72-839a-32f3e6694b63 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:09.327866 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40779464-464c-4555-96f4-ef1cff9c6d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:09.328474 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40779464-464c-4555-96f4-ef1cff9c6d67 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:09.328756 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40779464-464c-4555-96f4-ef1cff9c6d67 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:09.329034 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40779464-464c-4555-96f4-ef1cff9c6d67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:09.329847 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40779464-464c-4555-96f4-ef1cff9c6d67 None admin] RBAC: Adding query filter params (role.id=2bd1979f53bb4853a50dd5bee6aa0782, scope.project.id=f3b90869933b4b20a5ca1fee6006b44c, user.id=a1fe605d4b944c04a09bb8261f523c64) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:33:09.330160 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40779464-464c-4555-96f4-ef1cff9c6d67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:09.331571 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40779464-464c-4555-96f4-ef1cff9c6d67 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:09.337230 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 584/1161] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jun 15 14:33:09 2022] GET /identity/v3/role_assignments?user.id=a1fe605d4b944c04a09bb8261f523c64&scope.project.id=f3b90869933b4b20a5ca1fee6006b44c&role.id=2bd1979f53bb4853a50dd5bee6aa0782 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:33:11.449367 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8a7846-0727-4ad8-a40f-1445870b8a54 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:11.449925 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8a7846-0727-4ad8-a40f-1445870b8a54 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:11.449925 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8a7846-0727-4ad8-a40f-1445870b8a54 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:11.451263 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 578/1162] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:33:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:33:11.456336 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff618316-2cba-4fce-8543-30f9b3917d8e None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:11.456756 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff618316-2cba-4fce-8543-30f9b3917d8e None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:11.456893 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff618316-2cba-4fce-8543-30f9b3917d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:11.473751 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-ff618316-2cba-4fce-8543-30f9b3917d8e None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:33:11.474926 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-ff618316-2cba-4fce-8543-30f9b3917d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:33:11.539196 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 585/1163] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:33:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:33:11.562268 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8a7846-0727-4ad8-a40f-1445870b8a54 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:11.592196 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:11.661882 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8a7846-0727-4ad8-a40f-1445870b8a54 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:11.696106 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2302dc76-d789-49c7-a76d-929ff94a62c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:11.696751 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2302dc76-d789-49c7-a76d-929ff94a62c3 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:11.697036 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2302dc76-d789-49c7-a76d-929ff94a62c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:11.697311 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2302dc76-d789-49c7-a76d-929ff94a62c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:11.762939 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2302dc76-d789-49c7-a76d-929ff94a62c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:11.764411 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2302dc76-d789-49c7-a76d-929ff94a62c3 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:11.792611 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:11.860054 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 579/1164] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jun 15 14:33:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:33:11.866021 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff618316-2cba-4fce-8543-30f9b3917d8e None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:11.954212 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:11.963309 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff618316-2cba-4fce-8543-30f9b3917d8e None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:11.996966 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5757fedf-8a96-4c4b-b526-5d7b0fa43462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:11.997752 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5757fedf-8a96-4c4b-b526-5d7b0fa43462 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:11.998036 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5757fedf-8a96-4c4b-b526-5d7b0fa43462 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:11.998314 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5757fedf-8a96-4c4b-b526-5d7b0fa43462 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:12.004817 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5757fedf-8a96-4c4b-b526-5d7b0fa43462 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:12.006235 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5757fedf-8a96-4c4b-b526-5d7b0fa43462 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:12.020827 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-5757fedf-8a96-4c4b-b526-5d7b0fa43462 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 15 14:33:12.020827 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:33:12.020827 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 15 14:33:12.020827 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 15 14:33:12.020827 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:33:12.020827 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:33:12.020827 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:33:12.020827 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:33:12.020827 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 15 14:33:12.020827 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:33:12.020827 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:33:12.020827 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:33:12.020827 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:33:12.020827 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:33:12.020827 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:33:12.020827 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:33:12.020827 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:33:12.022052 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:33:12.022052 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:33:12.022052 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:33:12.022052 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:33:12.022052 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:33:12.022052 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:33:12.022052 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:33:12.022052 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:33:12.022052 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:33:12.022052 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:33:12.022052 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:33:12.022052 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:33:12.022052 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:33:12.022052 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:33:12.022052 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:33:12.022052 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:33:12.023085 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:33:12.023085 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:33:12.023085 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:33:12.023085 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:33:12.023085 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:33:12.023085 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:33:12.023085 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:33:12.023085 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:33:12.023085 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:33:12.023085 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:33:12.023085 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 15 14:33:12.023085 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:33:12.023085 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 15 14:33:12.023085 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:33:12.023085 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 586/1165] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Jun 15 14:33:11 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:33:12.028868 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2302dc76-d789-49c7-a76d-929ff94a62c3 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:12.125549 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2302dc76-d789-49c7-a76d-929ff94a62c3 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:12.153530 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:12.159017 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25b6c76-4ca8-4179-af1b-31141098151f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:12.159661 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25b6c76-4ca8-4179-af1b-31141098151f None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:12.159941 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25b6c76-4ca8-4179-af1b-31141098151f None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:12.160245 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25b6c76-4ca8-4179-af1b-31141098151f None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:12.160954 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25b6c76-4ca8-4179-af1b-31141098151f None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:33:12.161258 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25b6c76-4ca8-4179-af1b-31141098151f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:12.162599 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25b6c76-4ca8-4179-af1b-31141098151f None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:12.169693 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 580/1166] 10.222.0.42 () {56 vars in 1115 bytes} [Wed Jun 15 14:33:12 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:33:14.262857 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f33f4e8-d1be-48fd-b488-1864e4d109b3 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:14.263489 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f33f4e8-d1be-48fd-b488-1864e4d109b3 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:14.263489 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f33f4e8-d1be-48fd-b488-1864e4d109b3 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:14.264809 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 587/1167] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:33:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:33:14.270014 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573b0b19-49fd-42d8-bf17-14a8d7c95fdb None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:14.270404 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573b0b19-49fd-42d8-bf17-14a8d7c95fdb None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:14.270568 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573b0b19-49fd-42d8-bf17-14a8d7c95fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:14.287732 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-573b0b19-49fd-42d8-bf17-14a8d7c95fdb None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:33:14.288938 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-573b0b19-49fd-42d8-bf17-14a8d7c95fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:33:14.353370 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 581/1168] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:33:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:33:14.376927 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f33f4e8-d1be-48fd-b488-1864e4d109b3 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:14.477494 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f33f4e8-d1be-48fd-b488-1864e4d109b3 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:14.509997 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27f5e8d-d30f-4a89-a070-a0c291c19b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:14.511283 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f5e8d-d30f-4a89-a070-a0c291c19b46 None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:14.511595 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f5e8d-d30f-4a89-a070-a0c291c19b46 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:14.511875 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f5e8d-d30f-4a89-a070-a0c291c19b46 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:14.512661 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27f5e8d-d30f-4a89-a070-a0c291c19b46 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:14.514063 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27f5e8d-d30f-4a89-a070-a0c291c19b46 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:14.582881 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 588/1169] 10.222.0.42 () {60 vars in 1138 bytes} [Wed Jun 15 14:33:14 2022] POST /identity/v3/domains => generated 267 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:33:16.732649 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fcc0f3-f4c0-4c59-a521-932619340409 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:16.733201 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fcc0f3-f4c0-4c59-a521-932619340409 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:16.733201 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fcc0f3-f4c0-4c59-a521-932619340409 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:16.734561 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 582/1170] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:33:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:33:16.739698 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934c6012-7a1b-4f8d-89fc-a08304e07176 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:16.740142 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934c6012-7a1b-4f8d-89fc-a08304e07176 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:16.740279 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934c6012-7a1b-4f8d-89fc-a08304e07176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:16.757314 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-934c6012-7a1b-4f8d-89fc-a08304e07176 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:33:16.758482 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-934c6012-7a1b-4f8d-89fc-a08304e07176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:33:16.761935 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:16.822266 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 589/1171] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:33:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:33:16.846230 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fcc0f3-f4c0-4c59-a521-932619340409 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:16.949109 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fcc0f3-f4c0-4c59-a521-932619340409 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:16.971804 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:16.982920 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6929fc-b56a-4bce-9640-79659c4d814d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:16.983763 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6929fc-b56a-4bce-9640-79659c4d814d None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:16.984047 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6929fc-b56a-4bce-9640-79659c4d814d None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:16.984360 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6929fc-b56a-4bce-9640-79659c4d814d None admin] PATH_INFO: `/v3/domains/0c541d85bb974dbdb5e330e403fce049` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:16.995699 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6929fc-b56a-4bce-9640-79659c4d814d None admin] RBAC: Authorizing `identity:get_domain(domain_id=0c541d85bb974dbdb5e330e403fce049)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:16.997305 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6929fc-b56a-4bce-9640-79659c4d814d None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:16.998918 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 583/1172] 10.222.0.42 () {56 vars in 1183 bytes} [Wed Jun 15 14:33:16 2022] GET /identity/v3/domains/0c541d85bb974dbdb5e330e403fce049 => generated 267 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:33:17.004714 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934c6012-7a1b-4f8d-89fc-a08304e07176 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:17.099059 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934c6012-7a1b-4f8d-89fc-a08304e07176 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:17.131753 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3b0ebd-1af5-421f-9c66-bfd808a4cdb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:17.132421 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3b0ebd-1af5-421f-9c66-bfd808a4cdb5 None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:17.132702 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3b0ebd-1af5-421f-9c66-bfd808a4cdb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:17.132996 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3b0ebd-1af5-421f-9c66-bfd808a4cdb5 None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:17.133687 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3b0ebd-1af5-421f-9c66-bfd808a4cdb5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:17.135099 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3b0ebd-1af5-421f-9c66-bfd808a4cdb5 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:17.144320 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:17.215658 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 590/1173] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Jun 15 14:33:17 2022] POST /identity/v3/projects => generated 361 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:33:19.346092 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f50bb-d360-4be0-b221-2a7affd4641e None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:19.346644 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f50bb-d360-4be0-b221-2a7affd4641e None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:19.346644 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f50bb-d360-4be0-b221-2a7affd4641e None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:19.348003 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 584/1174] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:33:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:33:19.353111 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c53fba-3835-47d3-8d89-ef1a0514317f None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:19.353506 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c53fba-3835-47d3-8d89-ef1a0514317f None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:19.353664 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c53fba-3835-47d3-8d89-ef1a0514317f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:19.371134 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-45c53fba-3835-47d3-8d89-ef1a0514317f None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:33:19.372321 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-45c53fba-3835-47d3-8d89-ef1a0514317f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:33:19.436940 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 591/1175] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:33:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:33:19.460964 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02f50bb-d360-4be0-b221-2a7affd4641e None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:19.560823 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02f50bb-d360-4be0-b221-2a7affd4641e None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:19.592825 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c5661e-caa0-4333-8612-aa6f31ecbc79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:19.593675 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c5661e-caa0-4333-8612-aa6f31ecbc79 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:19.593958 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c5661e-caa0-4333-8612-aa6f31ecbc79 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:19.594233 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c5661e-caa0-4333-8612-aa6f31ecbc79 None admin] PATH_INFO: `/v3/domains/0c541d85bb974dbdb5e330e403fce049` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:19.595386 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c5661e-caa0-4333-8612-aa6f31ecbc79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=0c541d85bb974dbdb5e330e403fce049)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:19.596898 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c5661e-caa0-4333-8612-aa6f31ecbc79 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:19.598451 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 585/1176] 10.222.0.42 () {56 vars in 1183 bytes} [Wed Jun 15 14:33:19 2022] GET /identity/v3/domains/0c541d85bb974dbdb5e330e403fce049 => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:33:19.604884 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c53fba-3835-47d3-8d89-ef1a0514317f None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:19.700645 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c53fba-3835-47d3-8d89-ef1a0514317f None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:19.722546 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:19.733948 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d940e6-2f0b-4d86-a905-ea9bd9f00bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:19.742012 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d940e6-2f0b-4d86-a905-ea9bd9f00bdb None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:19.742012 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d940e6-2f0b-4d86-a905-ea9bd9f00bdb None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:19.742012 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d940e6-2f0b-4d86-a905-ea9bd9f00bdb None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:19.742012 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d940e6-2f0b-4d86-a905-ea9bd9f00bdb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:19.742012 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d940e6-2f0b-4d86-a905-ea9bd9f00bdb None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:19.877863 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 592/1177] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Jun 15 14:33:19 2022] POST /identity/v3/users => generated 311 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:33:22.027598 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271441ed-4590-4c09-ae4f-43a67c8b5287 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:22.028165 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271441ed-4590-4c09-ae4f-43a67c8b5287 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:22.028165 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271441ed-4590-4c09-ae4f-43a67c8b5287 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:22.029522 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 586/1178] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:33:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:33:22.034630 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f60bde-bb52-40f3-9848-3276c7b52723 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:22.035019 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f60bde-bb52-40f3-9848-3276c7b52723 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:22.035188 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f60bde-bb52-40f3-9848-3276c7b52723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:22.052284 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-73f60bde-bb52-40f3-9848-3276c7b52723 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:33:22.053453 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-73f60bde-bb52-40f3-9848-3276c7b52723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:33:22.118133 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 593/1179] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:33:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:33:22.141919 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271441ed-4590-4c09-ae4f-43a67c8b5287 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:22.195070 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:22.241686 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271441ed-4590-4c09-ae4f-43a67c8b5287 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:22.273728 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229e6c33-f87c-49dc-b050-741cbba89045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:22.274491 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229e6c33-f87c-49dc-b050-741cbba89045 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:22.274765 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229e6c33-f87c-49dc-b050-741cbba89045 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:22.275032 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229e6c33-f87c-49dc-b050-741cbba89045 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:22.285359 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229e6c33-f87c-49dc-b050-741cbba89045 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:33:22.292918 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-229e6c33-f87c-49dc-b050-741cbba89045 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:33:22.292918 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:33:22.292918 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:33:22.292918 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:33:22.292918 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:33:22.292918 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:33:22.292918 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:33:22.292918 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:33:22.292918 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:33:22.292918 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:33:22.292918 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:33:22.292918 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:33:22.292918 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:33:22.292918 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:33:22.292918 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:33:22.292918 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:33:22.294133 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:33:22.294133 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:33:22.294133 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:33:22.294133 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:33:22.294133 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:33:22.294133 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:33:22.294133 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:33:22.294133 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:33:22.294133 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:33:22.294133 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:33:22.294133 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:33:22.294133 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:33:22.294133 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:33:22.294133 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:33:22.294133 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:33:22.294133 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:33:22.295069 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:33:22.295069 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:33:22.295069 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:33:22.295069 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:33:22.295069 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:33:22.295069 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:33:22.295069 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:33:22.295069 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:33:22.295069 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:33:22.295069 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:33:22.295069 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:33:22.295069 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:33:22.295069 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 587/1180] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jun 15 14:33:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:33:22.299982 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f60bde-bb52-40f3-9848-3276c7b52723 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:22.395875 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f60bde-bb52-40f3-9848-3276c7b52723 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:22.405865 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:22.429825 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9549189f-2a66-4fb0-8d6d-a3da7b39e7b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:22.430480 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9549189f-2a66-4fb0-8d6d-a3da7b39e7b1 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:22.430762 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9549189f-2a66-4fb0-8d6d-a3da7b39e7b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:22.431040 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9549189f-2a66-4fb0-8d6d-a3da7b39e7b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:22.431777 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9549189f-2a66-4fb0-8d6d-a3da7b39e7b1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:33:22.432082 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9549189f-2a66-4fb0-8d6d-a3da7b39e7b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:22.434106 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9549189f-2a66-4fb0-8d6d-a3da7b39e7b1 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:22.442331 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 594/1181] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jun 15 14:33:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:33:22.448173 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229e6c33-f87c-49dc-b050-741cbba89045 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:22.545797 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229e6c33-f87c-49dc-b050-741cbba89045 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:22.558783 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:22.578623 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169ca596-01ce-4d63-b170-b0508c28378c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:22.579533 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169ca596-01ce-4d63-b170-b0508c28378c None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:22.579813 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169ca596-01ce-4d63-b170-b0508c28378c None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:22.580103 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169ca596-01ce-4d63-b170-b0508c28378c None admin] PATH_INFO: `/v3/users/ef4ee472e58a40a2ae690e1fd6c83c72` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:22.593302 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169ca596-01ce-4d63-b170-b0508c28378c None admin] RBAC: Authorizing `identity:get_user(user_id=ef4ee472e58a40a2ae690e1fd6c83c72)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:22.594725 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169ca596-01ce-4d63-b170-b0508c28378c None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:22.596376 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 588/1182] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jun 15 14:33:22 2022] GET /identity/v3/users/ef4ee472e58a40a2ae690e1fd6c83c72 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:33:22.602148 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9549189f-2a66-4fb0-8d6d-a3da7b39e7b1 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:22.701312 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9549189f-2a66-4fb0-8d6d-a3da7b39e7b1 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:22.734060 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22137348-54a5-4eb7-a61c-3fd217fd1a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:22.734686 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22137348-54a5-4eb7-a61c-3fd217fd1a88 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:22.734965 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22137348-54a5-4eb7-a61c-3fd217fd1a88 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:22.735238 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22137348-54a5-4eb7-a61c-3fd217fd1a88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:22.771681 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:22.799633 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22137348-54a5-4eb7-a61c-3fd217fd1a88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:22.801027 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22137348-54a5-4eb7-a61c-3fd217fd1a88 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:22.893337 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 595/1183] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jun 15 14:33:22 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:33:22.899445 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169ca596-01ce-4d63-b170-b0508c28378c None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:22.995525 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169ca596-01ce-4d63-b170-b0508c28378c None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:23.028693 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7249c7-dd4c-4cfb-b69a-ea8d2a504f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:23.029522 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7249c7-dd4c-4cfb-b69a-ea8d2a504f18 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:23.029802 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7249c7-dd4c-4cfb-b69a-ea8d2a504f18 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:23.030073 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7249c7-dd4c-4cfb-b69a-ea8d2a504f18 None admin] PATH_INFO: `/v3/projects/d957cffdee72497286696e4e84d06633` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:23.031213 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7249c7-dd4c-4cfb-b69a-ea8d2a504f18 None admin] RBAC: Authorizing `identity:get_project(project_id=d957cffdee72497286696e4e84d06633)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:23.032644 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7249c7-dd4c-4cfb-b69a-ea8d2a504f18 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:23.034294 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 589/1184] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jun 15 14:33:22 2022] GET /identity/v3/projects/d957cffdee72497286696e4e84d06633 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:33:23.040393 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22137348-54a5-4eb7-a61c-3fd217fd1a88 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:23.106907 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:23.136295 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22137348-54a5-4eb7-a61c-3fd217fd1a88 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:23.170093 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529cd2e1-f5d4-44f6-a2ea-2a37fcb83760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:23.170724 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529cd2e1-f5d4-44f6-a2ea-2a37fcb83760 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:23.171007 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529cd2e1-f5d4-44f6-a2ea-2a37fcb83760 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:23.171285 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529cd2e1-f5d4-44f6-a2ea-2a37fcb83760 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:23.172119 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529cd2e1-f5d4-44f6-a2ea-2a37fcb83760 None admin] RBAC: Adding query filter params (role.id=2bd1979f53bb4853a50dd5bee6aa0782, scope.project.id=d957cffdee72497286696e4e84d06633, user.id=ef4ee472e58a40a2ae690e1fd6c83c72) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:33:23.172437 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529cd2e1-f5d4-44f6-a2ea-2a37fcb83760 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:23.173861 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529cd2e1-f5d4-44f6-a2ea-2a37fcb83760 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:23.179146 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 596/1185] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jun 15 14:33:23 2022] GET /identity/v3/role_assignments?user.id=ef4ee472e58a40a2ae690e1fd6c83c72&scope.project.id=d957cffdee72497286696e4e84d06633&role.id=2bd1979f53bb4853a50dd5bee6aa0782 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:33:25.242514 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9f6fe8-97ad-4f68-8daf-4c9c87a726cb None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:25.243038 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9f6fe8-97ad-4f68-8daf-4c9c87a726cb None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:25.243038 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9f6fe8-97ad-4f68-8daf-4c9c87a726cb None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:25.244440 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 590/1186] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:33:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:33:25.250471 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e67fbc6-53ab-4915-8fc6-3e9b4ae9f05a None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:25.250861 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e67fbc6-53ab-4915-8fc6-3e9b4ae9f05a None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:25.251027 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e67fbc6-53ab-4915-8fc6-3e9b4ae9f05a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:25.267338 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-2e67fbc6-53ab-4915-8fc6-3e9b4ae9f05a None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:33:25.268544 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-2e67fbc6-53ab-4915-8fc6-3e9b4ae9f05a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:33:25.335991 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 597/1187] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:33:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:33:25.358644 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9f6fe8-97ad-4f68-8daf-4c9c87a726cb None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:25.398652 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:25.456408 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9f6fe8-97ad-4f68-8daf-4c9c87a726cb None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:25.488560 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c42ea49-41e5-45ae-bad4-dbcadcedbd30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:25.489328 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42ea49-41e5-45ae-bad4-dbcadcedbd30 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:25.489609 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42ea49-41e5-45ae-bad4-dbcadcedbd30 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:25.489879 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42ea49-41e5-45ae-bad4-dbcadcedbd30 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:25.500293 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c42ea49-41e5-45ae-bad4-dbcadcedbd30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:33:25.507754 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-0c42ea49-41e5-45ae-bad4-dbcadcedbd30 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:33:25.507754 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:33:25.507754 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:33:25.507754 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:33:25.507754 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:33:25.507754 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:33:25.507754 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:33:25.507754 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:33:25.507754 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:33:25.507754 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:33:25.507754 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:33:25.507754 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:33:25.507754 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:33:25.507754 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:33:25.507754 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:33:25.507754 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:33:25.508869 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:33:25.508869 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:33:25.508869 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:33:25.508869 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:33:25.508869 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:33:25.508869 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:33:25.508869 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:33:25.508869 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:33:25.508869 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:33:25.508869 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:33:25.508869 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:33:25.508869 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:33:25.508869 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:33:25.508869 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:33:25.508869 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:33:25.508869 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:33:25.509916 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:33:25.509916 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:33:25.509916 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:33:25.509916 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:33:25.509916 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:33:25.509916 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:33:25.509916 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:33:25.509916 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:33:25.509916 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:33:25.509916 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:33:25.509916 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:33:25.509916 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:33:25.509916 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 591/1188] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jun 15 14:33:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:33:25.516209 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e67fbc6-53ab-4915-8fc6-3e9b4ae9f05a None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:25.577174 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:25.613222 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e67fbc6-53ab-4915-8fc6-3e9b4ae9f05a None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:25.646618 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef98e1f4-30f3-4bb2-a6b8-b43dc5cb91dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:25.647274 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef98e1f4-30f3-4bb2-a6b8-b43dc5cb91dd None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:25.647560 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef98e1f4-30f3-4bb2-a6b8-b43dc5cb91dd None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:25.647836 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef98e1f4-30f3-4bb2-a6b8-b43dc5cb91dd None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:25.648859 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef98e1f4-30f3-4bb2-a6b8-b43dc5cb91dd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:33:25.649171 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef98e1f4-30f3-4bb2-a6b8-b43dc5cb91dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:25.650522 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef98e1f4-30f3-4bb2-a6b8-b43dc5cb91dd None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:25.658988 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 598/1189] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jun 15 14:33:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:33:25.664771 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c42ea49-41e5-45ae-bad4-dbcadcedbd30 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:25.758453 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:25.762720 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c42ea49-41e5-45ae-bad4-dbcadcedbd30 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:25.795694 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe9bc41-a0e2-43b1-8725-5b4f26571d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:25.796567 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe9bc41-a0e2-43b1-8725-5b4f26571d59 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:25.796855 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe9bc41-a0e2-43b1-8725-5b4f26571d59 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:25.797134 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe9bc41-a0e2-43b1-8725-5b4f26571d59 None admin] PATH_INFO: `/v3/users/ef4ee472e58a40a2ae690e1fd6c83c72` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:25.798311 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe9bc41-a0e2-43b1-8725-5b4f26571d59 None admin] RBAC: Authorizing `identity:get_user(user_id=ef4ee472e58a40a2ae690e1fd6c83c72)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:25.799733 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe9bc41-a0e2-43b1-8725-5b4f26571d59 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:25.801379 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 592/1190] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jun 15 14:33:25 2022] GET /identity/v3/users/ef4ee472e58a40a2ae690e1fd6c83c72 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:33:25.807089 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef98e1f4-30f3-4bb2-a6b8-b43dc5cb91dd None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:25.903404 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef98e1f4-30f3-4bb2-a6b8-b43dc5cb91dd None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:25.927185 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:25.935726 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccde97b-30d5-461d-bb73-cc5474fbd28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:25.936596 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccde97b-30d5-461d-bb73-cc5474fbd28a None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:25.936883 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccde97b-30d5-461d-bb73-cc5474fbd28a None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:25.937159 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccde97b-30d5-461d-bb73-cc5474fbd28a None admin] PATH_INFO: `/v3/projects/d957cffdee72497286696e4e84d06633` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:25.938333 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccde97b-30d5-461d-bb73-cc5474fbd28a None admin] RBAC: Authorizing `identity:get_project(project_id=d957cffdee72497286696e4e84d06633)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:25.939748 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccde97b-30d5-461d-bb73-cc5474fbd28a None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:25.941557 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 599/1191] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jun 15 14:33:25 2022] GET /identity/v3/projects/d957cffdee72497286696e4e84d06633 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:33:25.947488 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe9bc41-a0e2-43b1-8725-5b4f26571d59 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:26.048217 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe9bc41-a0e2-43b1-8725-5b4f26571d59 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:26.081565 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bb7881-2c29-409b-98a8-62f5df0b69b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:26.082437 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bb7881-2c29-409b-98a8-62f5df0b69b9 None admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:26.082717 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bb7881-2c29-409b-98a8-62f5df0b69b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:26.082992 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bb7881-2c29-409b-98a8-62f5df0b69b9 None admin] PATH_INFO: `/v3/projects/d957cffdee72497286696e4e84d06633/users/ef4ee472e58a40a2ae690e1fd6c83c72/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:26.084973 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bb7881-2c29-409b-98a8-62f5df0b69b9 None admin] RBAC: Authorizing `identity:create_grant(project_id=d957cffdee72497286696e4e84d06633, user_id=ef4ee472e58a40a2ae690e1fd6c83c72, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:26.089643 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bb7881-2c29-409b-98a8-62f5df0b69b9 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:26.204035 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 593/1192] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Jun 15 14:33:25 2022] PUT /identity/v3/projects/d957cffdee72497286696e4e84d06633/users/ef4ee472e58a40a2ae690e1fd6c83c72/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:33:28.489450 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca12c211-a6f5-4950-a7ef-7c1607b8d615 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:28.489980 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca12c211-a6f5-4950-a7ef-7c1607b8d615 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:28.490106 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca12c211-a6f5-4950-a7ef-7c1607b8d615 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:28.491391 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 600/1193] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:33:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:33:28.498449 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b7eeb-28ce-4907-af62-995bac69f18f None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:28.498884 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b7eeb-28ce-4907-af62-995bac69f18f None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:28.499010 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b7eeb-28ce-4907-af62-995bac69f18f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:28.516227 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-237b7eeb-28ce-4907-af62-995bac69f18f None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:33:28.517406 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-237b7eeb-28ce-4907-af62-995bac69f18f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:33:28.536919 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:28.583600 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 594/1194] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:33:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:33:28.609445 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca12c211-a6f5-4950-a7ef-7c1607b8d615 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:28.710744 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca12c211-a6f5-4950-a7ef-7c1607b8d615 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:28.744009 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e51a126-8565-40d8-932a-10533fc115fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:28.744747 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e51a126-8565-40d8-932a-10533fc115fd None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:28.745026 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e51a126-8565-40d8-932a-10533fc115fd None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:28.745296 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e51a126-8565-40d8-932a-10533fc115fd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:28.756732 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e51a126-8565-40d8-932a-10533fc115fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:33:28.764242 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-4e51a126-8565-40d8-932a-10533fc115fd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:33:28.764242 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:33:28.764242 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:33:28.764242 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:33:28.764242 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:33:28.764242 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:33:28.764242 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:33:28.764242 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:33:28.764242 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:33:28.764242 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:33:28.764242 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:33:28.764242 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:33:28.764242 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:33:28.764242 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:33:28.764242 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:33:28.764242 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:33:28.765371 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:33:28.765371 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:33:28.765371 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:33:28.765371 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:33:28.765371 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:33:28.765371 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:33:28.765371 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:33:28.765371 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:33:28.765371 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:33:28.765371 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:33:28.765371 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:33:28.765371 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:33:28.765371 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:33:28.765371 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:33:28.765371 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:33:28.765371 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:33:28.766436 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:33:28.766436 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:33:28.766436 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:33:28.766436 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:33:28.766436 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:33:28.766436 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:33:28.766436 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:33:28.766436 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:33:28.766436 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:33:28.766436 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:33:28.766436 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:33:28.766436 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:33:28.766436 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 601/1195] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jun 15 14:33:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:33:28.771388 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237b7eeb-28ce-4907-af62-995bac69f18f None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:28.866317 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237b7eeb-28ce-4907-af62-995bac69f18f None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:28.898910 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e7c9c-a434-4dcd-bf8d-c56fcd018634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:28.899557 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e7c9c-a434-4dcd-bf8d-c56fcd018634 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:28.899841 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e7c9c-a434-4dcd-bf8d-c56fcd018634 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:28.900137 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e7c9c-a434-4dcd-bf8d-c56fcd018634 None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:28.900870 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9e7c9c-a434-4dcd-bf8d-c56fcd018634 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:33:28.901174 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9e7c9c-a434-4dcd-bf8d-c56fcd018634 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:28.902544 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9e7c9c-a434-4dcd-bf8d-c56fcd018634 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:28.910343 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 595/1196] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jun 15 14:33:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:33:28.916103 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e51a126-8565-40d8-932a-10533fc115fd None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:28.971499 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:29.014541 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e51a126-8565-40d8-932a-10533fc115fd None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:29.047936 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8369aa-93d5-4b06-a6d0-e4a79bf33c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:29.048872 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8369aa-93d5-4b06-a6d0-e4a79bf33c2a None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:29.049157 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8369aa-93d5-4b06-a6d0-e4a79bf33c2a None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:29.049428 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8369aa-93d5-4b06-a6d0-e4a79bf33c2a None admin] PATH_INFO: `/v3/users/ef4ee472e58a40a2ae690e1fd6c83c72` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:29.050575 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8369aa-93d5-4b06-a6d0-e4a79bf33c2a None admin] RBAC: Authorizing `identity:get_user(user_id=ef4ee472e58a40a2ae690e1fd6c83c72)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:29.051979 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8369aa-93d5-4b06-a6d0-e4a79bf33c2a None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:29.053612 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 602/1197] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jun 15 14:33:28 2022] GET /identity/v3/users/ef4ee472e58a40a2ae690e1fd6c83c72 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:33:29.059702 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e7c9c-a434-4dcd-bf8d-c56fcd018634 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:29.078626 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:29.157553 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e7c9c-a434-4dcd-bf8d-c56fcd018634 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:29.189986 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dcd69b-985f-4e22-ba39-05566bc8abd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:29.190622 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dcd69b-985f-4e22-ba39-05566bc8abd6 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:29.190905 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dcd69b-985f-4e22-ba39-05566bc8abd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:29.191181 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dcd69b-985f-4e22-ba39-05566bc8abd6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:29.255443 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dcd69b-985f-4e22-ba39-05566bc8abd6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:29.256904 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dcd69b-985f-4e22-ba39-05566bc8abd6 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:29.274335 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:29.352044 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 596/1198] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jun 15 14:33:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:33:29.357945 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8369aa-93d5-4b06-a6d0-e4a79bf33c2a None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:29.453467 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8369aa-93d5-4b06-a6d0-e4a79bf33c2a None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:29.475506 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:29.486514 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6338daa9-3a82-487a-90c3-49b604599c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:29.487344 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338daa9-3a82-487a-90c3-49b604599c35 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:29.487621 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338daa9-3a82-487a-90c3-49b604599c35 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:29.489158 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338daa9-3a82-487a-90c3-49b604599c35 None admin] PATH_INFO: `/v3/projects/d957cffdee72497286696e4e84d06633` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:29.489158 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6338daa9-3a82-487a-90c3-49b604599c35 None admin] RBAC: Authorizing `identity:get_project(project_id=d957cffdee72497286696e4e84d06633)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:29.490659 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6338daa9-3a82-487a-90c3-49b604599c35 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:29.492355 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 603/1199] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jun 15 14:33:29 2022] GET /identity/v3/projects/d957cffdee72497286696e4e84d06633 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:33:29.498588 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dcd69b-985f-4e22-ba39-05566bc8abd6 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:29.594767 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dcd69b-985f-4e22-ba39-05566bc8abd6 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:29.611230 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:29.626872 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618111c1-74fc-444f-9cbc-6be4678b1097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:29.627515 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618111c1-74fc-444f-9cbc-6be4678b1097 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:29.627797 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618111c1-74fc-444f-9cbc-6be4678b1097 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:29.628071 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618111c1-74fc-444f-9cbc-6be4678b1097 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:29.628935 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618111c1-74fc-444f-9cbc-6be4678b1097 None admin] RBAC: Adding query filter params (role.id=2bd1979f53bb4853a50dd5bee6aa0782, scope.project.id=d957cffdee72497286696e4e84d06633, user.id=ef4ee472e58a40a2ae690e1fd6c83c72) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:33:29.629248 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618111c1-74fc-444f-9cbc-6be4678b1097 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:29.630663 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618111c1-74fc-444f-9cbc-6be4678b1097 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:29.636297 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 597/1200] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jun 15 14:33:29 2022] GET /identity/v3/role_assignments?user.id=ef4ee472e58a40a2ae690e1fd6c83c72&scope.project.id=d957cffdee72497286696e4e84d06633&role.id=2bd1979f53bb4853a50dd5bee6aa0782 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:33:39.518196 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=5 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:39.638515 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=9 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:43.013455 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a5ce3-1ec8-4ae8-bea3-54489365fab7 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:43.014042 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a5ce3-1ec8-4ae8-bea3-54489365fab7 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:43.014042 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a5ce3-1ec8-4ae8-bea3-54489365fab7 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:43.015570 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 604/1201] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:34:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:43.020778 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff8210-25a1-435b-a70a-66b0c7260861 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:43.021176 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff8210-25a1-435b-a70a-66b0c7260861 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:43.021333 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff8210-25a1-435b-a70a-66b0c7260861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:43.039020 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-83ff8210-25a1-435b-a70a-66b0c7260861 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:43.040264 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-83ff8210-25a1-435b-a70a-66b0c7260861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:43.107434 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 598/1202] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:34:43.132630 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689a5ce3-1ec8-4ae8-bea3-54489365fab7 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:43.234657 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689a5ce3-1ec8-4ae8-bea3-54489365fab7 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:43.268465 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4bdc86-1eb9-49cd-8b19-a9f352385696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:43.269182 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4bdc86-1eb9-49cd-8b19-a9f352385696 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:43.269460 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4bdc86-1eb9-49cd-8b19-a9f352385696 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:43.269730 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4bdc86-1eb9-49cd-8b19-a9f352385696 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:43.279871 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4bdc86-1eb9-49cd-8b19-a9f352385696 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:43.281320 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4bdc86-1eb9-49cd-8b19-a9f352385696 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:43.299122 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-ed4bdc86-1eb9-49cd-8b19-a9f352385696 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:34:43.299122 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:34:43.299122 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:34:43.299122 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:34:43.299122 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:34:43.299122 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:34:43.299122 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:34:43.299122 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:34:43.299122 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:34:43.299122 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:34:43.299122 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:34:43.299122 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:34:43.299122 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:34:43.299122 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:34:43.299122 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:34:43.300351 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:34:43.300351 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:34:43.300351 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:34:43.300351 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:34:43.300351 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:34:43.300351 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:34:43.300351 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:34:43.300351 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:34:43.300351 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:34:43.300351 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:34:43.300351 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:34:43.300351 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:34:43.300351 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:34:43.300351 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:34:43.300351 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:34:43.300351 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:34:43.300351 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:34:43.301549 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:34:43.301549 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:34:43.301549 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:34:43.301549 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:34:43.301549 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 605/1203] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:34:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:34:43.306454 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ff8210-25a1-435b-a70a-66b0c7260861 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:43.402982 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ff8210-25a1-435b-a70a-66b0c7260861 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:43.431188 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:43.437578 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a1a3bb-5452-4a71-aa38-659ff469a9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:43.442190 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a1a3bb-5452-4a71-aa38-659ff469a9d3 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:43.442190 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a1a3bb-5452-4a71-aa38-659ff469a9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:43.442190 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a1a3bb-5452-4a71-aa38-659ff469a9d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:43.442190 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a1a3bb-5452-4a71-aa38-659ff469a9d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:43.442190 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a1a3bb-5452-4a71-aa38-659ff469a9d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:43.442190 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a1a3bb-5452-4a71-aa38-659ff469a9d3 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:43.452888 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 599/1204] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:34:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:43.458955 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4bdc86-1eb9-49cd-8b19-a9f352385696 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:43.511862 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:43.559639 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4bdc86-1eb9-49cd-8b19-a9f352385696 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:43.592565 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bb9603-e921-4dbc-9355-11d5c9f583ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:43.593229 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb9603-e921-4dbc-9355-11d5c9f583ea None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:43.593508 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb9603-e921-4dbc-9355-11d5c9f583ea None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:43.593780 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb9603-e921-4dbc-9355-11d5c9f583ea None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:43.594482 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bb9603-e921-4dbc-9355-11d5c9f583ea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:43.595899 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bb9603-e921-4dbc-9355-11d5c9f583ea None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:43.668400 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 606/1205] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Jun 15 14:34:43 2022] POST /identity/v3/users => generated 251 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:34:45.799942 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bdb8c0-a5d5-4aba-95be-d8c75ac246f0 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:45.800524 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bdb8c0-a5d5-4aba-95be-d8c75ac246f0 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:45.800524 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bdb8c0-a5d5-4aba-95be-d8c75ac246f0 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:45.801892 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 600/1206] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:34:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:45.807137 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc99a715-59e0-41f9-876f-80b4ae05965b None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:45.807544 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc99a715-59e0-41f9-876f-80b4ae05965b None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:45.807643 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc99a715-59e0-41f9-876f-80b4ae05965b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:45.825433 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-bc99a715-59e0-41f9-876f-80b4ae05965b None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:45.826624 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-bc99a715-59e0-41f9-876f-80b4ae05965b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:45.894168 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 607/1207] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:34:45.918725 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bdb8c0-a5d5-4aba-95be-d8c75ac246f0 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:46.017571 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bdb8c0-a5d5-4aba-95be-d8c75ac246f0 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:46.051679 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1e60ed-311a-4226-b04c-56941e515aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:46.052562 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e60ed-311a-4226-b04c-56941e515aa8 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:46.052843 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e60ed-311a-4226-b04c-56941e515aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:46.053120 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e60ed-311a-4226-b04c-56941e515aa8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:46.063761 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1e60ed-311a-4226-b04c-56941e515aa8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:34:46.071412 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-fb1e60ed-311a-4226-b04c-56941e515aa8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:34:46.071412 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:34:46.071412 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:34:46.071412 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:34:46.071412 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:34:46.071412 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:34:46.071412 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:34:46.071412 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:34:46.071412 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:34:46.071412 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:34:46.071412 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:34:46.071412 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:34:46.071412 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:34:46.071412 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:34:46.071412 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:34:46.071412 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:34:46.075782 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:34:46.075782 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:34:46.075782 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:34:46.075782 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:34:46.075782 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:34:46.075782 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:34:46.075782 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:34:46.075782 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:34:46.075782 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:34:46.075782 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:34:46.075782 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:34:46.075782 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:34:46.075782 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:34:46.075782 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:34:46.075782 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:34:46.075782 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:34:46.076696 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:34:46.076696 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:34:46.076696 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:34:46.076696 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:34:46.076696 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:34:46.076696 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:34:46.076696 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:34:46.076696 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:34:46.076696 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:34:46.076696 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:34:46.076696 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:34:46.076696 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:34:46.076696 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 601/1208] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Jun 15 14:34:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:34:46.078943 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc99a715-59e0-41f9-876f-80b4ae05965b None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:46.117884 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:46.174918 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc99a715-59e0-41f9-876f-80b4ae05965b None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:46.208724 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e428b-cbf9-4bce-b0aa-4301fe0291b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:46.209383 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e428b-cbf9-4bce-b0aa-4301fe0291b1 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:46.209667 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e428b-cbf9-4bce-b0aa-4301fe0291b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:46.209943 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e428b-cbf9-4bce-b0aa-4301fe0291b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:46.210681 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185e428b-cbf9-4bce-b0aa-4301fe0291b1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:46.210989 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185e428b-cbf9-4bce-b0aa-4301fe0291b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:46.212381 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185e428b-cbf9-4bce-b0aa-4301fe0291b1 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:46.220975 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 608/1209] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Jun 15 14:34:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:46.226833 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1e60ed-311a-4226-b04c-56941e515aa8 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:46.278417 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:46.326126 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1e60ed-311a-4226-b04c-56941e515aa8 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:46.360274 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791071d0-7aa5-4149-aba6-8930dfde2ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:46.361050 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791071d0-7aa5-4149-aba6-8930dfde2ff4 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:46.361334 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791071d0-7aa5-4149-aba6-8930dfde2ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:46.361610 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791071d0-7aa5-4149-aba6-8930dfde2ff4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:46.372688 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791071d0-7aa5-4149-aba6-8930dfde2ff4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:46.374197 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791071d0-7aa5-4149-aba6-8930dfde2ff4 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:46.391939 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-791071d0-7aa5-4149-aba6-8930dfde2ff4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:34:46.391939 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:34:46.391939 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:34:46.391939 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:34:46.391939 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:34:46.391939 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:34:46.391939 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:34:46.391939 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:34:46.391939 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:34:46.391939 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:34:46.391939 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:34:46.391939 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:34:46.391939 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:34:46.391939 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:34:46.391939 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:34:46.393424 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:34:46.393424 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:34:46.393424 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:34:46.393424 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:34:46.393424 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:34:46.393424 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:34:46.393424 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:34:46.393424 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:34:46.393424 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:34:46.393424 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:34:46.393424 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:34:46.393424 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:34:46.393424 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:34:46.393424 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:34:46.393424 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:34:46.393424 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:34:46.393424 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:34:46.394475 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:34:46.394475 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:34:46.394475 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:34:46.394475 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:34:46.394475 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 602/1210] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:34:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:34:46.399333 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e428b-cbf9-4bce-b0aa-4301fe0291b1 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:46.498957 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:46.501818 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e428b-cbf9-4bce-b0aa-4301fe0291b1 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:46.535012 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c1fa20-ffc5-4df0-b1af-7cd1a66ada66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:46.535661 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c1fa20-ffc5-4df0-b1af-7cd1a66ada66 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:46.535941 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c1fa20-ffc5-4df0-b1af-7cd1a66ada66 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:46.536246 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c1fa20-ffc5-4df0-b1af-7cd1a66ada66 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:46.536959 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c1fa20-ffc5-4df0-b1af-7cd1a66ada66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:46.537284 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c1fa20-ffc5-4df0-b1af-7cd1a66ada66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:46.538648 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c1fa20-ffc5-4df0-b1af-7cd1a66ada66 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:46.552367 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 609/1211] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:34:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:46.557069 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791071d0-7aa5-4149-aba6-8930dfde2ff4 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:46.655115 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791071d0-7aa5-4149-aba6-8930dfde2ff4 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:46.656595 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:46.688893 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4663bb10-7aeb-4f2e-b99a-e4f6b83fe53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:46.689682 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4663bb10-7aeb-4f2e-b99a-e4f6b83fe53a None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:46.689966 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4663bb10-7aeb-4f2e-b99a-e4f6b83fe53a None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:46.690242 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4663bb10-7aeb-4f2e-b99a-e4f6b83fe53a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:46.697096 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4663bb10-7aeb-4f2e-b99a-e4f6b83fe53a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:46.698552 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4663bb10-7aeb-4f2e-b99a-e4f6b83fe53a None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:46.715311 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-4663bb10-7aeb-4f2e-b99a-e4f6b83fe53a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 15 14:34:46.715311 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:34:46.715311 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:34:46.715311 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:34:46.715311 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:34:46.715311 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:34:46.715311 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:34:46.715311 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:34:46.715311 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:34:46.715311 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:34:46.715311 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:34:46.715311 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:34:46.715311 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:34:46.715311 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:34:46.715311 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:34:46.715311 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:34:46.716351 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:34:46.716351 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:34:46.716351 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:34:46.716351 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:34:46.716351 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:34:46.716351 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:34:46.716351 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:34:46.716351 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:34:46.716351 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:34:46.716351 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:34:46.716351 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:34:46.716351 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:34:46.716351 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:34:46.716351 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:34:46.716351 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:34:46.716351 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:34:46.716351 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:34:46.716351 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:34:46.717393 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:34:46.717393 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:34:46.717393 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:34:46.717393 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:34:46.717393 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:34:46.717393 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:34:46.717393 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:34:46.717393 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:34:46.717393 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 15 14:34:46.717393 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:34:46.717393 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 603/1212] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:34:46 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:34:46.722345 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c1fa20-ffc5-4df0-b1af-7cd1a66ada66 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:46.824615 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c1fa20-ffc5-4df0-b1af-7cd1a66ada66 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:46.858422 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf308eee-f193-4cb6-b6b7-e8b39759bc8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:46.859144 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf308eee-f193-4cb6-b6b7-e8b39759bc8b None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:46.859430 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf308eee-f193-4cb6-b6b7-e8b39759bc8b None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:46.859707 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf308eee-f193-4cb6-b6b7-e8b39759bc8b None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:46.860564 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf308eee-f193-4cb6-b6b7-e8b39759bc8b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:46.860874 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf308eee-f193-4cb6-b6b7-e8b39759bc8b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:46.862289 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf308eee-f193-4cb6-b6b7-e8b39759bc8b None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:46.883931 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 610/1213] 10.222.0.42 () {56 vars in 1143 bytes} [Wed Jun 15 14:34:46 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:46.885925 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4663bb10-7aeb-4f2e-b99a-e4f6b83fe53a None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:46.984888 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4663bb10-7aeb-4f2e-b99a-e4f6b83fe53a None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:47.018052 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de1903b-114a-47df-97e0-0aba71a2a942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:47.018685 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de1903b-114a-47df-97e0-0aba71a2a942 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:47.018963 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de1903b-114a-47df-97e0-0aba71a2a942 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:47.019236 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de1903b-114a-47df-97e0-0aba71a2a942 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:47.050011 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:47.084653 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de1903b-114a-47df-97e0-0aba71a2a942 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:47.086123 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de1903b-114a-47df-97e0-0aba71a2a942 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:47.184258 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 604/1214] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jun 15 14:34:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:34:47.186338 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf308eee-f193-4cb6-b6b7-e8b39759bc8b None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:47.206029 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:47.286311 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf308eee-f193-4cb6-b6b7-e8b39759bc8b None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:47.320450 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759426b3-96f2-435e-8be1-0f4ee4f452fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:47.321234 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759426b3-96f2-435e-8be1-0f4ee4f452fc None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:47.321514 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759426b3-96f2-435e-8be1-0f4ee4f452fc None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:47.321805 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759426b3-96f2-435e-8be1-0f4ee4f452fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:47.332247 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759426b3-96f2-435e-8be1-0f4ee4f452fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:47.333701 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759426b3-96f2-435e-8be1-0f4ee4f452fc None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:47.351306 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-759426b3-96f2-435e-8be1-0f4ee4f452fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:34:47.351306 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:34:47.351306 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:34:47.351306 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:34:47.351306 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:34:47.351306 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:34:47.351306 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:34:47.351306 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:34:47.351306 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:34:47.351306 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:34:47.351306 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:34:47.351306 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:34:47.351306 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:34:47.351306 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:34:47.351306 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:34:47.352483 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:34:47.352483 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:34:47.352483 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:34:47.352483 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:34:47.352483 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:34:47.352483 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:34:47.352483 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:34:47.352483 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:34:47.352483 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:34:47.352483 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:34:47.352483 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:34:47.352483 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:34:47.352483 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:34:47.352483 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:34:47.352483 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:34:47.352483 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:34:47.352483 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:34:47.353460 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:34:47.353460 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:34:47.353460 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:34:47.353460 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:34:47.353460 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 611/1215] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:34:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:34:47.358326 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de1903b-114a-47df-97e0-0aba71a2a942 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:47.417809 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:47.455919 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de1903b-114a-47df-97e0-0aba71a2a942 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:47.489742 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a6363a-6e30-4447-88c7-3494833a247d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:47.490398 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a6363a-6e30-4447-88c7-3494833a247d None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:47.490682 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a6363a-6e30-4447-88c7-3494833a247d None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:47.490962 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a6363a-6e30-4447-88c7-3494833a247d None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:47.491678 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a6363a-6e30-4447-88c7-3494833a247d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:47.491987 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a6363a-6e30-4447-88c7-3494833a247d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:47.493376 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a6363a-6e30-4447-88c7-3494833a247d None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:47.505247 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 605/1216] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:34:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:47.511011 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759426b3-96f2-435e-8be1-0f4ee4f452fc None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:47.581675 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:47.610948 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759426b3-96f2-435e-8be1-0f4ee4f452fc None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:47.644370 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d897ba3a-e1d9-44af-9d24-f8376c03ecc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:47.645143 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d897ba3a-e1d9-44af-9d24-f8376c03ecc0 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:47.645421 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d897ba3a-e1d9-44af-9d24-f8376c03ecc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:47.645693 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d897ba3a-e1d9-44af-9d24-f8376c03ecc0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:47.652590 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d897ba3a-e1d9-44af-9d24-f8376c03ecc0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:47.654012 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d897ba3a-e1d9-44af-9d24-f8376c03ecc0 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:47.667308 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-d897ba3a-e1d9-44af-9d24-f8376c03ecc0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:34:47.667308 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:34:47.667308 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:34:47.667308 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:34:47.667308 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:34:47.667308 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:34:47.667308 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:34:47.667308 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:34:47.667308 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:34:47.667308 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:34:47.667308 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:34:47.667308 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:34:47.667308 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:34:47.667308 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:34:47.667308 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:34:47.669833 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:34:47.669833 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:34:47.669833 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:34:47.669833 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:34:47.669833 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:34:47.669833 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:34:47.669833 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:34:47.669833 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:34:47.669833 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:34:47.669833 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:34:47.669833 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:34:47.669833 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:34:47.669833 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:34:47.669833 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:34:47.669833 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:34:47.669833 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:34:47.669833 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:34:47.670967 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:34:47.670967 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:34:47.670967 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:34:47.670967 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:34:47.670967 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:34:47.670967 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:34:47.670967 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:34:47.670967 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:34:47.670967 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 612/1217] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jun 15 14:34:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:34:47.674754 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a6363a-6e30-4447-88c7-3494833a247d None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:47.777938 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a6363a-6e30-4447-88c7-3494833a247d None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:47.800727 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:47.812946 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3cd71b-f49a-46d9-acf1-12bd29d66b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:47.813533 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3cd71b-f49a-46d9-acf1-12bd29d66b84 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:47.813812 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3cd71b-f49a-46d9-acf1-12bd29d66b84 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:47.814082 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3cd71b-f49a-46d9-acf1-12bd29d66b84 None admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:47.814853 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3cd71b-f49a-46d9-acf1-12bd29d66b84 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:47.815159 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3cd71b-f49a-46d9-acf1-12bd29d66b84 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:47.816613 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3cd71b-f49a-46d9-acf1-12bd29d66b84 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:47.828291 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 606/1218] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jun 15 14:34:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:47.834453 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d897ba3a-e1d9-44af-9d24-f8376c03ecc0 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:47.934710 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d897ba3a-e1d9-44af-9d24-f8376c03ecc0 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:47.962483 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:47.968977 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204c5518-e55c-4b22-81ab-f6b72d69417a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:47.969624 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204c5518-e55c-4b22-81ab-f6b72d69417a None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:47.969908 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204c5518-e55c-4b22-81ab-f6b72d69417a None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:47.970183 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204c5518-e55c-4b22-81ab-f6b72d69417a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:47.971001 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204c5518-e55c-4b22-81ab-f6b72d69417a None admin] RBAC: Adding query filter params (role.id=305a1ca2e6f740eba7dc696849e933eb, scope.project.id=3094ff8966f149bda784da6a51b1358f, user.id=2bb6671c19464424a6e9cd4519ac93e2) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:47.971342 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204c5518-e55c-4b22-81ab-f6b72d69417a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:47.972793 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204c5518-e55c-4b22-81ab-f6b72d69417a None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:47.978264 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 613/1219] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jun 15 14:34:47 2022] GET /identity/v3/role_assignments?user.id=2bb6671c19464424a6e9cd4519ac93e2&scope.project.id=3094ff8966f149bda784da6a51b1358f&role.id=305a1ca2e6f740eba7dc696849e933eb => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:50.057109 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a0723-a60c-468d-a15e-a50edfbae63a None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:50.057686 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a0723-a60c-468d-a15e-a50edfbae63a None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:50.057686 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a0723-a60c-468d-a15e-a50edfbae63a None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:50.059775 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 607/1220] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:34:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:50.065021 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a839e871-f4b5-41d9-8a52-547f3fc1a09f None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:50.065419 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a839e871-f4b5-41d9-8a52-547f3fc1a09f None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:50.065606 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a839e871-f4b5-41d9-8a52-547f3fc1a09f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:50.082354 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-a839e871-f4b5-41d9-8a52-547f3fc1a09f None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:50.087923 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a839e871-f4b5-41d9-8a52-547f3fc1a09f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:50.149946 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 614/1221] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:34:50.173125 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6a0723-a60c-468d-a15e-a50edfbae63a None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:50.271540 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6a0723-a60c-468d-a15e-a50edfbae63a None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:50.305544 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de51770-9053-4664-8a92-1f1647430291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:50.306375 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de51770-9053-4664-8a92-1f1647430291 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:50.306618 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de51770-9053-4664-8a92-1f1647430291 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:50.306897 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de51770-9053-4664-8a92-1f1647430291 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:50.319013 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de51770-9053-4664-8a92-1f1647430291 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:34:50.325257 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-9de51770-9053-4664-8a92-1f1647430291 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:34:50.325257 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:34:50.325257 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:34:50.325257 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:34:50.325257 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:34:50.325257 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:34:50.325257 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:34:50.325257 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:34:50.325257 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:34:50.325257 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:34:50.325257 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:34:50.325257 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:34:50.325257 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:34:50.325257 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:34:50.325257 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:34:50.325257 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:34:50.326414 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:34:50.326414 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:34:50.326414 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:34:50.326414 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:34:50.326414 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:34:50.326414 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:34:50.326414 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:34:50.326414 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:34:50.326414 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:34:50.326414 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:34:50.326414 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:34:50.326414 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:34:50.326414 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:34:50.326414 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:34:50.326414 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:34:50.326414 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:34:50.327466 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:34:50.327466 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:34:50.327466 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:34:50.327466 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:34:50.327466 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:34:50.327466 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:34:50.327466 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:34:50.327466 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:34:50.327466 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:34:50.327466 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:34:50.327466 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:34:50.327466 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:34:50.327466 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 608/1222] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Jun 15 14:34:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:34:50.332779 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a839e871-f4b5-41d9-8a52-547f3fc1a09f None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:50.430399 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a839e871-f4b5-41d9-8a52-547f3fc1a09f None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:50.452559 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:50.462767 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288b05ed-4dd2-4bcb-a60f-d351bb2c8334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:50.463424 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288b05ed-4dd2-4bcb-a60f-d351bb2c8334 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:50.463708 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288b05ed-4dd2-4bcb-a60f-d351bb2c8334 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:50.463987 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288b05ed-4dd2-4bcb-a60f-d351bb2c8334 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:50.465417 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288b05ed-4dd2-4bcb-a60f-d351bb2c8334 None admin] RBAC: Adding query filter params (name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:50.465723 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288b05ed-4dd2-4bcb-a60f-d351bb2c8334 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:50.467079 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288b05ed-4dd2-4bcb-a60f-d351bb2c8334 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:50.474732 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 615/1223] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Jun 15 14:34:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:50.481224 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de51770-9053-4664-8a92-1f1647430291 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:50.525285 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:50.581888 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de51770-9053-4664-8a92-1f1647430291 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:50.614555 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7686f5-a335-4b1b-b1e8-956eb6ca55c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:50.615331 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7686f5-a335-4b1b-b1e8-956eb6ca55c3 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:50.615608 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7686f5-a335-4b1b-b1e8-956eb6ca55c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:50.615879 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7686f5-a335-4b1b-b1e8-956eb6ca55c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:50.626805 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7686f5-a335-4b1b-b1e8-956eb6ca55c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:50.630336 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7686f5-a335-4b1b-b1e8-956eb6ca55c3 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:50.644927 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-ae7686f5-a335-4b1b-b1e8-956eb6ca55c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:34:50.644927 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:34:50.644927 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:34:50.644927 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:34:50.644927 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:34:50.644927 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:34:50.644927 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:34:50.644927 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:34:50.644927 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:34:50.644927 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:34:50.644927 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:34:50.644927 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:34:50.644927 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:34:50.644927 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:34:50.644927 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:34:50.645983 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:34:50.645983 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:34:50.645983 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:34:50.645983 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:34:50.645983 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:34:50.645983 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:34:50.645983 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:34:50.645983 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:34:50.645983 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:34:50.645983 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:34:50.645983 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:34:50.645983 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:34:50.645983 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:34:50.645983 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:34:50.645983 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:34:50.645983 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:34:50.645983 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:34:50.647145 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:34:50.647145 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:34:50.647145 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:34:50.647145 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:34:50.647145 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 609/1224] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:34:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:34:50.652731 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288b05ed-4dd2-4bcb-a60f-d351bb2c8334 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:50.753126 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288b05ed-4dd2-4bcb-a60f-d351bb2c8334 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:50.787094 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66196cba-636a-46ef-8717-1285ed371313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:50.787737 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66196cba-636a-46ef-8717-1285ed371313 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:50.788017 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66196cba-636a-46ef-8717-1285ed371313 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:50.788317 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66196cba-636a-46ef-8717-1285ed371313 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:50.789023 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66196cba-636a-46ef-8717-1285ed371313 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:50.789325 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66196cba-636a-46ef-8717-1285ed371313 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:50.790663 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66196cba-636a-46ef-8717-1285ed371313 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:50.807286 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 616/1225] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:34:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:50.813651 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7686f5-a335-4b1b-b1e8-956eb6ca55c3 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:50.906578 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:50.913066 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7686f5-a335-4b1b-b1e8-956eb6ca55c3 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:50.946316 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e1a09f-66ec-4c7d-b0c3-257d6db83593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:50.947103 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e1a09f-66ec-4c7d-b0c3-257d6db83593 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:50.947386 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e1a09f-66ec-4c7d-b0c3-257d6db83593 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:50.947662 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e1a09f-66ec-4c7d-b0c3-257d6db83593 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:50.955420 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e1a09f-66ec-4c7d-b0c3-257d6db83593 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:50.956870 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e1a09f-66ec-4c7d-b0c3-257d6db83593 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:50.971336 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-74e1a09f-66ec-4c7d-b0c3-257d6db83593 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 15 14:34:50.971336 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:34:50.971336 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:34:50.971336 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:34:50.971336 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:34:50.971336 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:34:50.971336 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:34:50.971336 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:34:50.971336 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:34:50.971336 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:34:50.971336 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:34:50.971336 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:34:50.971336 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:34:50.971336 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:34:50.971336 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:34:50.971336 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:34:50.972497 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:34:50.972497 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:34:50.972497 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:34:50.972497 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:34:50.972497 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:34:50.972497 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:34:50.972497 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:34:50.972497 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:34:50.972497 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:34:50.972497 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:34:50.972497 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:34:50.972497 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:34:50.972497 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:34:50.972497 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:34:50.972497 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:34:50.972497 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:34:50.972497 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:34:50.972497 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:34:50.973445 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:34:50.973445 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:34:50.973445 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:34:50.973445 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:34:50.973445 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:34:50.973445 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:34:50.973445 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:34:50.973445 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:34:50.973445 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 15 14:34:50.973445 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:34:50.973445 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 610/1226] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:34:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:34:50.978459 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66196cba-636a-46ef-8717-1285ed371313 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:51.024622 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:51.078201 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66196cba-636a-46ef-8717-1285ed371313 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:51.111846 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe712ae5-f4d3-4517-8576-1199873dd638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:51.112531 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe712ae5-f4d3-4517-8576-1199873dd638 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:51.112812 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe712ae5-f4d3-4517-8576-1199873dd638 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:51.113085 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe712ae5-f4d3-4517-8576-1199873dd638 None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:51.113904 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe712ae5-f4d3-4517-8576-1199873dd638 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:51.114210 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe712ae5-f4d3-4517-8576-1199873dd638 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:51.115618 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe712ae5-f4d3-4517-8576-1199873dd638 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:51.132235 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 617/1227] 10.222.0.42 () {56 vars in 1143 bytes} [Wed Jun 15 14:34:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:51.138701 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e1a09f-66ec-4c7d-b0c3-257d6db83593 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:51.237121 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e1a09f-66ec-4c7d-b0c3-257d6db83593 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:51.270806 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce443d4-e092-43f6-8898-29e969e5c1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:51.271577 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce443d4-e092-43f6-8898-29e969e5c1d4 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:51.271859 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce443d4-e092-43f6-8898-29e969e5c1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:51.272251 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce443d4-e092-43f6-8898-29e969e5c1d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:51.282999 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce443d4-e092-43f6-8898-29e969e5c1d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:51.284450 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce443d4-e092-43f6-8898-29e969e5c1d4 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:51.301149 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-fce443d4-e092-43f6-8898-29e969e5c1d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:34:51.301149 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:34:51.301149 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:34:51.301149 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:34:51.301149 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:34:51.301149 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:34:51.301149 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:34:51.301149 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:34:51.301149 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:34:51.301149 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:34:51.301149 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:34:51.301149 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:34:51.301149 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:34:51.301149 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:34:51.301149 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:34:51.303734 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:34:51.303734 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:34:51.303734 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:34:51.303734 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:34:51.303734 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:34:51.303734 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:34:51.303734 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:34:51.303734 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:34:51.303734 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:34:51.303734 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:34:51.303734 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:34:51.303734 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:34:51.303734 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:34:51.303734 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:34:51.303734 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:34:51.303734 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:34:51.303734 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:34:51.304710 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:34:51.304710 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:34:51.304710 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:34:51.304710 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:34:51.304710 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 611/1228] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:34:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:34:51.308943 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe712ae5-f4d3-4517-8576-1199873dd638 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:51.408560 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe712ae5-f4d3-4517-8576-1199873dd638 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:51.410908 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:51.441783 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4cd366-de77-4380-a9d8-6ba728c3c588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:51.442433 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4cd366-de77-4380-a9d8-6ba728c3c588 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:51.442713 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4cd366-de77-4380-a9d8-6ba728c3c588 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:51.442985 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4cd366-de77-4380-a9d8-6ba728c3c588 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:51.443690 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4cd366-de77-4380-a9d8-6ba728c3c588 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:51.443992 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4cd366-de77-4380-a9d8-6ba728c3c588 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:51.445367 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4cd366-de77-4380-a9d8-6ba728c3c588 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:51.456970 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 618/1229] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:34:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:51.462814 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce443d4-e092-43f6-8898-29e969e5c1d4 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:51.472124 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:51.562181 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce443d4-e092-43f6-8898-29e969e5c1d4 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:51.594931 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871d1ab9-2b61-47aa-a16a-f9dac47bae1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:51.595704 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871d1ab9-2b61-47aa-a16a-f9dac47bae1d None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:51.595982 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871d1ab9-2b61-47aa-a16a-f9dac47bae1d None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:51.596279 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871d1ab9-2b61-47aa-a16a-f9dac47bae1d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:51.602722 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871d1ab9-2b61-47aa-a16a-f9dac47bae1d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:51.604248 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871d1ab9-2b61-47aa-a16a-f9dac47bae1d None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:51.619567 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-871d1ab9-2b61-47aa-a16a-f9dac47bae1d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:34:51.619567 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:34:51.619567 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:34:51.619567 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:34:51.619567 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:34:51.619567 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:34:51.619567 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:34:51.619567 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:34:51.619567 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:34:51.619567 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:34:51.619567 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:34:51.619567 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:34:51.619567 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:34:51.619567 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:34:51.619567 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:34:51.621273 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:34:51.621273 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:34:51.621273 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:34:51.621273 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:34:51.621273 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:34:51.621273 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:34:51.621273 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:34:51.621273 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:34:51.621273 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:34:51.621273 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:34:51.621273 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:34:51.621273 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:34:51.621273 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:34:51.621273 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:34:51.621273 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:34:51.621273 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:34:51.621273 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:34:51.633470 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:34:51.633470 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:34:51.633470 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:34:51.633470 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:34:51.633470 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:34:51.633470 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:34:51.633470 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:34:51.633470 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:34:51.633470 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 612/1230] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jun 15 14:34:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:34:51.634071 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4cd366-de77-4380-a9d8-6ba728c3c588 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:51.724555 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4cd366-de77-4380-a9d8-6ba728c3c588 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:51.757439 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bb53eb-db48-4ad7-a13b-463052d12958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:51.758081 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb53eb-db48-4ad7-a13b-463052d12958 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:51.758361 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb53eb-db48-4ad7-a13b-463052d12958 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:51.758636 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb53eb-db48-4ad7-a13b-463052d12958 None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:51.759450 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bb53eb-db48-4ad7-a13b-463052d12958 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:51.759757 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bb53eb-db48-4ad7-a13b-463052d12958 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:51.761496 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bb53eb-db48-4ad7-a13b-463052d12958 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:51.774264 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 619/1231] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jun 15 14:34:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:51.780476 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871d1ab9-2b61-47aa-a16a-f9dac47bae1d None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:51.845147 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:51.878707 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871d1ab9-2b61-47aa-a16a-f9dac47bae1d None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:51.911515 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1a3e95-ad4a-4fc3-bbf2-a8fe6dece43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:51.912409 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a3e95-ad4a-4fc3-bbf2-a8fe6dece43b None admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:51.912688 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a3e95-ad4a-4fc3-bbf2-a8fe6dece43b None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:51.912959 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a3e95-ad4a-4fc3-bbf2-a8fe6dece43b None admin] PATH_INFO: `/v3/projects/3094ff8966f149bda784da6a51b1358f/users/2bb6671c19464424a6e9cd4519ac93e2/roles/305a1ca2e6f740eba7dc696849e933eb` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:51.930130 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1a3e95-ad4a-4fc3-bbf2-a8fe6dece43b None admin] RBAC: Authorizing `identity:create_grant(project_id=3094ff8966f149bda784da6a51b1358f, user_id=2bb6671c19464424a6e9cd4519ac93e2, role_id=305a1ca2e6f740eba7dc696849e933eb)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:51.932161 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1a3e95-ad4a-4fc3-bbf2-a8fe6dece43b None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:52.008533 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 613/1232] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Jun 15 14:34:51 2022] PUT /identity/v3/projects/3094ff8966f149bda784da6a51b1358f/users/2bb6671c19464424a6e9cd4519ac93e2/roles/305a1ca2e6f740eba7dc696849e933eb => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:34:54.141617 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d65c2-6e87-4068-8ee1-ff35263d561a None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:54.142193 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d65c2-6e87-4068-8ee1-ff35263d561a None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:54.142193 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d65c2-6e87-4068-8ee1-ff35263d561a None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:54.143530 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 620/1233] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:34:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:54.149137 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0239b5d-4e63-482c-9aeb-c0210f5c2380 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:54.149627 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0239b5d-4e63-482c-9aeb-c0210f5c2380 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:54.156949 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0239b5d-4e63-482c-9aeb-c0210f5c2380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:54.166513 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-c0239b5d-4e63-482c-9aeb-c0210f5c2380 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:54.168466 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-c0239b5d-4e63-482c-9aeb-c0210f5c2380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:54.234882 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 614/1234] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:34:54.260046 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d65c2-6e87-4068-8ee1-ff35263d561a None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:54.286364 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:54.361749 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d65c2-6e87-4068-8ee1-ff35263d561a None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:54.396913 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f36c96-f094-4d7b-9238-48453d6a39ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:54.397623 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f36c96-f094-4d7b-9238-48453d6a39ec None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:54.397903 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f36c96-f094-4d7b-9238-48453d6a39ec None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:54.398430 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f36c96-f094-4d7b-9238-48453d6a39ec None admin] PATH_INFO: `/v3/roles/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:54.408864 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f36c96-f094-4d7b-9238-48453d6a39ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:34:54.416369 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-66f36c96-f094-4d7b-9238-48453d6a39ec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:34:54.416369 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:34:54.416369 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:34:54.416369 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:34:54.416369 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:34:54.416369 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:34:54.416369 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:34:54.416369 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:34:54.416369 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:34:54.416369 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:34:54.416369 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:34:54.416369 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:34:54.416369 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:34:54.416369 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:34:54.416369 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:34:54.416369 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:34:54.418379 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:34:54.418379 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:34:54.418379 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:34:54.418379 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:34:54.418379 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:34:54.418379 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:34:54.418379 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:34:54.418379 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:34:54.418379 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:34:54.418379 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:34:54.418379 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:34:54.418379 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:34:54.418379 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:34:54.418379 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:34:54.418379 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:34:54.418379 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:34:54.419747 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:34:54.419747 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:34:54.419747 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:34:54.419747 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:34:54.419747 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:34:54.419747 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:34:54.419747 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:34:54.419747 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:34:54.419747 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:34:54.419747 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:34:54.419747 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:34:54.419747 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:34:54.419747 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 621/1235] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Jun 15 14:34:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:34:54.424189 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0239b5d-4e63-482c-9aeb-c0210f5c2380 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:54.470389 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:54.524543 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0239b5d-4e63-482c-9aeb-c0210f5c2380 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:54.558518 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d92e0d-574d-455a-9fe5-0bcc6770df85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:54.559171 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d92e0d-574d-455a-9fe5-0bcc6770df85 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:54.559450 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d92e0d-574d-455a-9fe5-0bcc6770df85 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:54.559724 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d92e0d-574d-455a-9fe5-0bcc6770df85 None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:54.560499 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d92e0d-574d-455a-9fe5-0bcc6770df85 None admin] RBAC: Adding query filter params (name=service) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:54.560804 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d92e0d-574d-455a-9fe5-0bcc6770df85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:54.562164 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d92e0d-574d-455a-9fe5-0bcc6770df85 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:54.570559 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 615/1236] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Jun 15 14:34:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:54.577080 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f36c96-f094-4d7b-9238-48453d6a39ec None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:54.661614 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:54.677079 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f36c96-f094-4d7b-9238-48453d6a39ec None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:54.709354 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df7628f-08c0-4c5d-ac39-9ce5343ae76b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:54.710133 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df7628f-08c0-4c5d-ac39-9ce5343ae76b None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:54.710414 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df7628f-08c0-4c5d-ac39-9ce5343ae76b None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:54.710689 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df7628f-08c0-4c5d-ac39-9ce5343ae76b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:54.722826 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df7628f-08c0-4c5d-ac39-9ce5343ae76b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:54.724300 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df7628f-08c0-4c5d-ac39-9ce5343ae76b None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:54.743297 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-5df7628f-08c0-4c5d-ac39-9ce5343ae76b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:34:54.743297 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:34:54.743297 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:34:54.743297 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:34:54.743297 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:34:54.743297 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:34:54.743297 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:34:54.743297 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:34:54.743297 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:34:54.743297 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:34:54.743297 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:34:54.743297 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:34:54.743297 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:34:54.743297 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:34:54.743297 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:34:54.744310 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:34:54.744310 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:34:54.744310 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:34:54.744310 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:34:54.744310 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:34:54.744310 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:34:54.744310 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:34:54.744310 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:34:54.744310 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:34:54.744310 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:34:54.744310 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:34:54.744310 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:34:54.744310 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:34:54.744310 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:34:54.744310 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:34:54.744310 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:34:54.744310 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:34:54.745211 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:34:54.745211 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:34:54.745211 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:34:54.745211 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:34:54.751174 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 622/1237] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:34:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:34:54.751290 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d92e0d-574d-455a-9fe5-0bcc6770df85 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:54.855770 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d92e0d-574d-455a-9fe5-0bcc6770df85 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:54.865878 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:54.890950 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710741e4-d16d-4b9c-bbd9-c12c3e05f914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:54.891667 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710741e4-d16d-4b9c-bbd9-c12c3e05f914 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:54.891943 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710741e4-d16d-4b9c-bbd9-c12c3e05f914 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:54.892260 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710741e4-d16d-4b9c-bbd9-c12c3e05f914 None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:54.893008 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710741e4-d16d-4b9c-bbd9-c12c3e05f914 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:54.893316 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710741e4-d16d-4b9c-bbd9-c12c3e05f914 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:54.894819 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710741e4-d16d-4b9c-bbd9-c12c3e05f914 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:54.907323 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 616/1238] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:34:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:54.913223 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df7628f-08c0-4c5d-ac39-9ce5343ae76b None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:55.017517 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df7628f-08c0-4c5d-ac39-9ce5343ae76b None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:55.050049 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9f0cdc-2f08-4b01-9c45-657bbd20c373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:55.050842 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f0cdc-2f08-4b01-9c45-657bbd20c373 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:55.051125 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f0cdc-2f08-4b01-9c45-657bbd20c373 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:55.051403 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f0cdc-2f08-4b01-9c45-657bbd20c373 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:55.054069 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:55.059408 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9f0cdc-2f08-4b01-9c45-657bbd20c373 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:55.060865 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9f0cdc-2f08-4b01-9c45-657bbd20c373 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:55.075403 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-1c9f0cdc-2f08-4b01-9c45-657bbd20c373 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 15 14:34:55.075403 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:34:55.075403 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:34:55.075403 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:34:55.075403 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:34:55.075403 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:34:55.075403 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:34:55.075403 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:34:55.075403 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:34:55.075403 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:34:55.075403 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:34:55.075403 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:34:55.075403 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:34:55.075403 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:34:55.075403 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:34:55.075403 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:34:55.076617 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:34:55.076617 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:34:55.076617 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:34:55.076617 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:34:55.076617 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:34:55.076617 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:34:55.076617 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:34:55.076617 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:34:55.076617 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:34:55.076617 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:34:55.076617 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:34:55.076617 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:34:55.076617 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:34:55.076617 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:34:55.076617 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:34:55.076617 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:34:55.076617 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:34:55.076617 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:34:55.077945 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:34:55.077945 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:34:55.077945 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:34:55.077945 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:34:55.077945 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:34:55.077945 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:34:55.077945 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:34:55.077945 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:34:55.077945 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 15 14:34:55.077945 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:34:55.077945 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 623/1239] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:34:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:34:55.082482 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710741e4-d16d-4b9c-bbd9-c12c3e05f914 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:55.179269 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710741e4-d16d-4b9c-bbd9-c12c3e05f914 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:55.212595 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7df292-8338-4393-a032-fa08c1ea5b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:55.213257 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7df292-8338-4393-a032-fa08c1ea5b26 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:55.213538 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7df292-8338-4393-a032-fa08c1ea5b26 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:55.213813 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7df292-8338-4393-a032-fa08c1ea5b26 None admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:55.214680 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7df292-8338-4393-a032-fa08c1ea5b26 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:55.214989 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7df292-8338-4393-a032-fa08c1ea5b26 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:55.216441 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7df292-8338-4393-a032-fa08c1ea5b26 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:55.233794 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 617/1240] 10.222.0.42 () {56 vars in 1143 bytes} [Wed Jun 15 14:34:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:55.239532 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9f0cdc-2f08-4b01-9c45-657bbd20c373 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:55.338344 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9f0cdc-2f08-4b01-9c45-657bbd20c373 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:55.372745 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7737a6a0-ad3a-4948-9c69-6efdf478e582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:55.373495 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7737a6a0-ad3a-4948-9c69-6efdf478e582 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:55.373781 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7737a6a0-ad3a-4948-9c69-6efdf478e582 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:55.374060 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7737a6a0-ad3a-4948-9c69-6efdf478e582 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:55.433910 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:55.440756 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7737a6a0-ad3a-4948-9c69-6efdf478e582 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:55.442213 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7737a6a0-ad3a-4948-9c69-6efdf478e582 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:55.540449 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 624/1241] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jun 15 14:34:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:34:55.547365 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7df292-8338-4393-a032-fa08c1ea5b26 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:55.575288 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:55.645200 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7df292-8338-4393-a032-fa08c1ea5b26 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:55.680599 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1419a9e6-39cc-46e1-8f1a-78f3fc3c2bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:55.686726 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1419a9e6-39cc-46e1-8f1a-78f3fc3c2bc8 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:55.686726 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1419a9e6-39cc-46e1-8f1a-78f3fc3c2bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:55.686726 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1419a9e6-39cc-46e1-8f1a-78f3fc3c2bc8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:55.692152 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1419a9e6-39cc-46e1-8f1a-78f3fc3c2bc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:55.693718 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1419a9e6-39cc-46e1-8f1a-78f3fc3c2bc8 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:55.710993 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-1419a9e6-39cc-46e1-8f1a-78f3fc3c2bc8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:34:55.710993 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:34:55.710993 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:34:55.710993 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:34:55.710993 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:34:55.710993 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:34:55.710993 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:34:55.710993 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:34:55.710993 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:34:55.710993 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:34:55.710993 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:34:55.710993 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:34:55.710993 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:34:55.710993 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:34:55.710993 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:34:55.716164 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:34:55.716164 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:34:55.716164 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:34:55.716164 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:34:55.716164 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:34:55.716164 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:34:55.716164 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:34:55.716164 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:34:55.716164 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:34:55.716164 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:34:55.716164 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:34:55.716164 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:34:55.716164 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:34:55.716164 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:34:55.716164 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:34:55.716164 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:34:55.716164 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:34:55.717105 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:34:55.717105 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:34:55.717105 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:34:55.717105 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:34:55.717105 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 618/1242] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:34:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:34:55.718891 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7737a6a0-ad3a-4948-9c69-6efdf478e582 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:55.809865 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:55.815385 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7737a6a0-ad3a-4948-9c69-6efdf478e582 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:55.847637 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91ffbf6-4348-47f7-b44f-b04aad7b1c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:55.848311 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91ffbf6-4348-47f7-b44f-b04aad7b1c39 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:55.848592 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91ffbf6-4348-47f7-b44f-b04aad7b1c39 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:55.848867 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91ffbf6-4348-47f7-b44f-b04aad7b1c39 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:55.849575 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91ffbf6-4348-47f7-b44f-b04aad7b1c39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:55.849881 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91ffbf6-4348-47f7-b44f-b04aad7b1c39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:55.851209 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91ffbf6-4348-47f7-b44f-b04aad7b1c39 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:55.862802 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 625/1243] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:34:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:55.868456 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1419a9e6-39cc-46e1-8f1a-78f3fc3c2bc8 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:55.945561 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:55.966228 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1419a9e6-39cc-46e1-8f1a-78f3fc3c2bc8 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:55.999067 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23ae2a0-3928-4326-b192-f7ff31acf8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:55.999845 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23ae2a0-3928-4326-b192-f7ff31acf8c5 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:56.000149 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23ae2a0-3928-4326-b192-f7ff31acf8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:56.000426 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23ae2a0-3928-4326-b192-f7ff31acf8c5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:56.006744 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23ae2a0-3928-4326-b192-f7ff31acf8c5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:56.008186 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23ae2a0-3928-4326-b192-f7ff31acf8c5 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:56.022288 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-e23ae2a0-3928-4326-b192-f7ff31acf8c5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:34:56.022288 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:34:56.022288 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:34:56.022288 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:34:56.022288 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:34:56.022288 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:34:56.022288 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:34:56.022288 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:34:56.022288 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:34:56.022288 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:34:56.022288 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:34:56.022288 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:34:56.022288 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:34:56.022288 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:34:56.022288 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:34:56.023660 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:34:56.023660 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:34:56.023660 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:34:56.023660 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:34:56.023660 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:34:56.023660 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:34:56.023660 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:34:56.023660 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:34:56.023660 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:34:56.023660 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:34:56.023660 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:34:56.023660 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:34:56.023660 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:34:56.023660 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:34:56.023660 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:34:56.023660 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:34:56.023660 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:34:56.029653 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:34:56.029653 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:34:56.029653 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:34:56.029653 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:34:56.029653 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:34:56.029653 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:34:56.029653 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:34:56.029653 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:34:56.029653 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 619/1244] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jun 15 14:34:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:34:56.030316 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91ffbf6-4348-47f7-b44f-b04aad7b1c39 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:56.128056 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91ffbf6-4348-47f7-b44f-b04aad7b1c39 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:56.161210 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09597302-c478-49fc-8a94-47b8797a407f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:56.161849 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09597302-c478-49fc-8a94-47b8797a407f None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:56.162126 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09597302-c478-49fc-8a94-47b8797a407f None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:56.162397 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09597302-c478-49fc-8a94-47b8797a407f None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:56.163174 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09597302-c478-49fc-8a94-47b8797a407f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:56.163479 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09597302-c478-49fc-8a94-47b8797a407f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:56.164907 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09597302-c478-49fc-8a94-47b8797a407f None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:56.176701 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 626/1245] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jun 15 14:34:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:56.182847 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23ae2a0-3928-4326-b192-f7ff31acf8c5 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:56.284308 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23ae2a0-3928-4326-b192-f7ff31acf8c5 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:56.317487 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381888ea-1b60-4453-b1cd-203adb10a4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:56.318139 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381888ea-1b60-4453-b1cd-203adb10a4f7 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:56.318431 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381888ea-1b60-4453-b1cd-203adb10a4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:56.318712 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381888ea-1b60-4453-b1cd-203adb10a4f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:56.319600 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381888ea-1b60-4453-b1cd-203adb10a4f7 None admin] RBAC: Adding query filter params (role.id=305a1ca2e6f740eba7dc696849e933eb, scope.project.id=3094ff8966f149bda784da6a51b1358f, user.id=2bb6671c19464424a6e9cd4519ac93e2) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:56.319876 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381888ea-1b60-4453-b1cd-203adb10a4f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:56.321341 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381888ea-1b60-4453-b1cd-203adb10a4f7 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:56.328288 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 620/1246] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jun 15 14:34:56 2022] GET /identity/v3/role_assignments?user.id=2bb6671c19464424a6e9cd4519ac93e2&scope.project.id=3094ff8966f149bda784da6a51b1358f&role.id=305a1ca2e6f740eba7dc696849e933eb => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:58.482196 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca14e9c-7386-42ac-ad66-a737d7a2e24d None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:58.482753 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca14e9c-7386-42ac-ad66-a737d7a2e24d None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:58.482863 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca14e9c-7386-42ac-ad66-a737d7a2e24d None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:58.484248 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 627/1247] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:34:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:34:58.489425 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b9ab4f-927d-459b-ba9f-1d00c08c095d None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:58.489895 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b9ab4f-927d-459b-ba9f-1d00c08c095d None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:58.489990 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b9ab4f-927d-459b-ba9f-1d00c08c095d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:58.495096 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:58.507461 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-86b9ab4f-927d-459b-ba9f-1d00c08c095d None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:58.508675 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-86b9ab4f-927d-459b-ba9f-1d00c08c095d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:58.576495 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 621/1248] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:34:58.601161 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca14e9c-7386-42ac-ad66-a737d7a2e24d None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:58.619787 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:58.703398 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca14e9c-7386-42ac-ad66-a737d7a2e24d None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:58.737535 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a40f59-82f8-457a-8456-c7cc5f84c66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:58.738255 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a40f59-82f8-457a-8456-c7cc5f84c66f None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:58.738560 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a40f59-82f8-457a-8456-c7cc5f84c66f None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:58.738840 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a40f59-82f8-457a-8456-c7cc5f84c66f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:58.750400 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a40f59-82f8-457a-8456-c7cc5f84c66f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:34:58.757781 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-48a40f59-82f8-457a-8456-c7cc5f84c66f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:34:58.757781 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:34:58.757781 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:34:58.757781 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:34:58.757781 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:34:58.757781 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:34:58.757781 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:34:58.757781 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:34:58.757781 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:34:58.757781 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:34:58.757781 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:34:58.757781 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:34:58.757781 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:34:58.757781 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:34:58.757781 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:34:58.757781 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:34:58.761292 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:34:58.761292 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:34:58.761292 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:34:58.761292 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:34:58.761292 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:34:58.761292 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:34:58.761292 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:34:58.761292 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:34:58.761292 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:34:58.761292 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:34:58.761292 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:34:58.761292 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:34:58.761292 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:34:58.761292 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:34:58.761292 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:34:58.761292 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:34:58.762492 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:34:58.762492 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:34:58.762492 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:34:58.762492 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:34:58.762492 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:34:58.762492 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:34:58.762492 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:34:58.762492 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:34:58.762492 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:34:58.762492 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:34:58.762492 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:34:58.762492 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:34:58.762492 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 628/1249] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jun 15 14:34:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:34:58.766055 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b9ab4f-927d-459b-ba9f-1d00c08c095d None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:58.863847 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b9ab4f-927d-459b-ba9f-1d00c08c095d None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:58.893067 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:58.897999 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beffd9ce-2e7d-484b-bae2-01bfebfabcc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:58.898924 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beffd9ce-2e7d-484b-bae2-01bfebfabcc9 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:58.899211 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beffd9ce-2e7d-484b-bae2-01bfebfabcc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:58.899491 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beffd9ce-2e7d-484b-bae2-01bfebfabcc9 None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:58.900273 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beffd9ce-2e7d-484b-bae2-01bfebfabcc9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:58.900587 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beffd9ce-2e7d-484b-bae2-01bfebfabcc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:58.902535 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beffd9ce-2e7d-484b-bae2-01bfebfabcc9 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:58.910617 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 622/1250] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jun 15 14:34:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:58.916521 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a40f59-82f8-457a-8456-c7cc5f84c66f None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:58.993356 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:59.016508 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a40f59-82f8-457a-8456-c7cc5f84c66f None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:59.050746 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfd12d8-cf59-42f9-8f01-dcbdba48377f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:59.051557 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfd12d8-cf59-42f9-8f01-dcbdba48377f None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:59.051836 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfd12d8-cf59-42f9-8f01-dcbdba48377f None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:59.052179 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfd12d8-cf59-42f9-8f01-dcbdba48377f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:59.062960 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfd12d8-cf59-42f9-8f01-dcbdba48377f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:59.064398 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfd12d8-cf59-42f9-8f01-dcbdba48377f None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:59.082547 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-2cfd12d8-cf59-42f9-8f01-dcbdba48377f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:34:59.082547 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:34:59.082547 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:34:59.082547 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:34:59.082547 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:34:59.082547 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:34:59.082547 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:34:59.082547 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:34:59.082547 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:34:59.082547 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:34:59.082547 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:34:59.082547 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:34:59.082547 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:34:59.082547 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:34:59.082547 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:34:59.087893 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:34:59.087893 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:34:59.087893 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:34:59.087893 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:34:59.087893 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:34:59.087893 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:34:59.087893 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:34:59.087893 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:34:59.087893 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:34:59.087893 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:34:59.087893 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:34:59.087893 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:34:59.087893 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:34:59.087893 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:34:59.087893 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:34:59.087893 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:34:59.087893 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:34:59.089094 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:34:59.089094 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:34:59.089094 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:34:59.089094 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:34:59.089094 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 629/1251] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:34:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:34:59.091251 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beffd9ce-2e7d-484b-bae2-01bfebfabcc9 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:59.190038 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beffd9ce-2e7d-484b-bae2-01bfebfabcc9 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:59.224030 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7290c1d-0172-4815-a63f-c7af3000e91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:59.224719 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7290c1d-0172-4815-a63f-c7af3000e91f None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:59.224997 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7290c1d-0172-4815-a63f-c7af3000e91f None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:59.225269 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7290c1d-0172-4815-a63f-c7af3000e91f None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:59.226193 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7290c1d-0172-4815-a63f-c7af3000e91f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:59.226498 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7290c1d-0172-4815-a63f-c7af3000e91f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:59.227860 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7290c1d-0172-4815-a63f-c7af3000e91f None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:59.240151 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 623/1252] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:34:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:59.245982 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfd12d8-cf59-42f9-8f01-dcbdba48377f None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:59.347654 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfd12d8-cf59-42f9-8f01-dcbdba48377f None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:59.373192 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:59.380750 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46e9d26-040c-4a59-9a2c-45ebf25fee68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:59.381532 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e9d26-040c-4a59-9a2c-45ebf25fee68 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:59.381817 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e9d26-040c-4a59-9a2c-45ebf25fee68 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:59.382091 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e9d26-040c-4a59-9a2c-45ebf25fee68 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:59.390113 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46e9d26-040c-4a59-9a2c-45ebf25fee68 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:59.391510 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46e9d26-040c-4a59-9a2c-45ebf25fee68 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:59.406727 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-a46e9d26-040c-4a59-9a2c-45ebf25fee68 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 15 14:34:59.406727 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:34:59.406727 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:34:59.406727 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:34:59.406727 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:34:59.406727 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:34:59.406727 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:34:59.406727 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:34:59.406727 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:34:59.406727 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:34:59.406727 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:34:59.406727 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:34:59.406727 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:34:59.406727 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:34:59.406727 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:34:59.406727 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:34:59.407835 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:34:59.407835 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:34:59.407835 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:34:59.407835 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:34:59.407835 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:34:59.407835 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:34:59.407835 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:34:59.407835 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:34:59.407835 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:34:59.407835 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:34:59.407835 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:34:59.407835 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:34:59.407835 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:34:59.407835 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:34:59.407835 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:34:59.407835 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:34:59.407835 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:34:59.407835 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:34:59.409188 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:34:59.409188 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:34:59.409188 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:34:59.409188 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:34:59.409188 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:34:59.409188 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:34:59.409188 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:34:59.409188 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:34:59.409188 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 15 14:34:59.409188 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:34:59.409188 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 630/1253] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:34:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:34:59.413761 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7290c1d-0172-4815-a63f-c7af3000e91f None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:59.468668 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:59.511162 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7290c1d-0172-4815-a63f-c7af3000e91f None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:59.545943 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe410fd-a095-4010-a969-8fd5c4791d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:59.546622 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe410fd-a095-4010-a969-8fd5c4791d14 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:59.546908 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe410fd-a095-4010-a969-8fd5c4791d14 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:59.547187 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe410fd-a095-4010-a969-8fd5c4791d14 None admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:59.548022 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe410fd-a095-4010-a969-8fd5c4791d14 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:59.548365 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe410fd-a095-4010-a969-8fd5c4791d14 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:59.549818 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe410fd-a095-4010-a969-8fd5c4791d14 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:59.567811 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 624/1254] 10.222.0.42 () {56 vars in 1143 bytes} [Wed Jun 15 14:34:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:59.573648 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46e9d26-040c-4a59-9a2c-45ebf25fee68 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:59.675049 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46e9d26-040c-4a59-9a2c-45ebf25fee68 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:59.709826 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be8143a-ffec-4601-a09c-3747f592302d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:59.710530 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be8143a-ffec-4601-a09c-3747f592302d None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:59.710829 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be8143a-ffec-4601-a09c-3747f592302d None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:59.711104 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be8143a-ffec-4601-a09c-3747f592302d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:59.765883 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:59.777874 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be8143a-ffec-4601-a09c-3747f592302d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:59.779348 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be8143a-ffec-4601-a09c-3747f592302d None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:59.876022 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 631/1255] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jun 15 14:34:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:34:59.882148 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe410fd-a095-4010-a969-8fd5c4791d14 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:59.979632 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe410fd-a095-4010-a969-8fd5c4791d14 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:59.993298 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:00.014465 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753a193e-c780-4845-ba39-f8380ef283cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:00.015245 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753a193e-c780-4845-ba39-f8380ef283cb None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:00.015526 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753a193e-c780-4845-ba39-f8380ef283cb None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:00.015800 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753a193e-c780-4845-ba39-f8380ef283cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:00.025776 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753a193e-c780-4845-ba39-f8380ef283cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:00.027242 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753a193e-c780-4845-ba39-f8380ef283cb None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:00.044586 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-753a193e-c780-4845-ba39-f8380ef283cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:00.044586 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:00.044586 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:00.044586 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:00.044586 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:00.044586 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:00.044586 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:00.044586 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:00.044586 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:00.044586 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:00.044586 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:00.044586 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:00.044586 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:35:00.044586 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:35:00.044586 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:35:00.051751 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:35:00.051751 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:00.051751 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:00.051751 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:00.051751 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:00.051751 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:00.051751 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:00.051751 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:00.051751 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:00.051751 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:00.051751 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:00.051751 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:00.051751 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:00.051751 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:00.051751 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:00.051751 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:00.051751 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:00.052700 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:35:00.052700 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:35:00.052700 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:00.052700 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:35:00.052700 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 625/1256] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:34:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:00.052993 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be8143a-ffec-4601-a09c-3747f592302d None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:00.138460 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:00.150141 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be8143a-ffec-4601-a09c-3747f592302d None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:00.184469 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caea6de3-ce28-45a1-82f3-d3cab5ee454c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:00.185142 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caea6de3-ce28-45a1-82f3-d3cab5ee454c None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:00.185429 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caea6de3-ce28-45a1-82f3-d3cab5ee454c None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:00.185713 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caea6de3-ce28-45a1-82f3-d3cab5ee454c None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:00.186433 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caea6de3-ce28-45a1-82f3-d3cab5ee454c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:00.186746 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caea6de3-ce28-45a1-82f3-d3cab5ee454c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:00.188128 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caea6de3-ce28-45a1-82f3-d3cab5ee454c None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:00.199428 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 632/1257] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:35:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:00.205072 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753a193e-c780-4845-ba39-f8380ef283cb None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:00.303411 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753a193e-c780-4845-ba39-f8380ef283cb None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:00.336416 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29885710-6a0f-46f4-9362-290eebc2865f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:00.337204 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29885710-6a0f-46f4-9362-290eebc2865f None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:00.337495 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29885710-6a0f-46f4-9362-290eebc2865f None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:00.337774 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29885710-6a0f-46f4-9362-290eebc2865f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:00.344943 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29885710-6a0f-46f4-9362-290eebc2865f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:00.346352 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29885710-6a0f-46f4-9362-290eebc2865f None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:00.360281 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-29885710-6a0f-46f4-9362-290eebc2865f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:35:00.360281 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:00.360281 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:00.360281 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:00.360281 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:00.360281 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:00.360281 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:00.360281 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:00.360281 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:00.360281 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:00.360281 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:00.360281 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:00.360281 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:35:00.360281 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:35:00.360281 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:35:00.362578 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:35:00.362578 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:00.362578 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:00.362578 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:00.362578 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:00.362578 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:00.362578 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:00.362578 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:00.362578 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:00.362578 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:00.362578 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:00.362578 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:00.362578 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:00.362578 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:00.362578 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:00.362578 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:00.362578 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:00.363693 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:35:00.363693 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:35:00.363693 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:35:00.363693 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:35:00.363693 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:35:00.363693 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:35:00.363693 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:35:00.363693 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:35:00.363693 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 626/1258] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jun 15 14:35:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:00.367502 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caea6de3-ce28-45a1-82f3-d3cab5ee454c None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:00.467798 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caea6de3-ce28-45a1-82f3-d3cab5ee454c None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:00.500983 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3151c1d5-7eed-46ae-9575-ff9db6f685b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:00.501633 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3151c1d5-7eed-46ae-9575-ff9db6f685b0 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:00.501916 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3151c1d5-7eed-46ae-9575-ff9db6f685b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:00.502193 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3151c1d5-7eed-46ae-9575-ff9db6f685b0 None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:00.502970 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3151c1d5-7eed-46ae-9575-ff9db6f685b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:00.503279 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3151c1d5-7eed-46ae-9575-ff9db6f685b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:00.504732 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3151c1d5-7eed-46ae-9575-ff9db6f685b0 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:00.518135 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 633/1259] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jun 15 14:35:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:00.524542 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29885710-6a0f-46f4-9362-290eebc2865f None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:00.552637 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:00.627916 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29885710-6a0f-46f4-9362-290eebc2865f None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:00.661498 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2161bc83-cb2a-4f2e-bee7-5a74ab103fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:00.662148 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2161bc83-cb2a-4f2e-bee7-5a74ab103fea None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:00.662429 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2161bc83-cb2a-4f2e-bee7-5a74ab103fea None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:00.662704 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2161bc83-cb2a-4f2e-bee7-5a74ab103fea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:00.663526 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2161bc83-cb2a-4f2e-bee7-5a74ab103fea None admin] RBAC: Adding query filter params (role.id=2bd1979f53bb4853a50dd5bee6aa0782, scope.project.id=3094ff8966f149bda784da6a51b1358f, user.id=2bb6671c19464424a6e9cd4519ac93e2) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:00.663839 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2161bc83-cb2a-4f2e-bee7-5a74ab103fea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:00.665297 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2161bc83-cb2a-4f2e-bee7-5a74ab103fea None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:00.671213 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 627/1260] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jun 15 14:35:00 2022] GET /identity/v3/role_assignments?user.id=2bb6671c19464424a6e9cd4519ac93e2&scope.project.id=3094ff8966f149bda784da6a51b1358f&role.id=2bd1979f53bb4853a50dd5bee6aa0782 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:02.751210 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50acfb72-2513-468d-b3fa-da2bc13da098 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:02.751783 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50acfb72-2513-468d-b3fa-da2bc13da098 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:02.751783 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50acfb72-2513-468d-b3fa-da2bc13da098 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:02.753148 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 634/1261] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:35:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:02.758324 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff839717-6a7c-4bb8-8244-70fcc2e57344 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:02.758740 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff839717-6a7c-4bb8-8244-70fcc2e57344 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:02.758856 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff839717-6a7c-4bb8-8244-70fcc2e57344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:02.776611 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-ff839717-6a7c-4bb8-8244-70fcc2e57344 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:02.777842 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-ff839717-6a7c-4bb8-8244-70fcc2e57344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:02.843688 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 628/1262] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:35:02.866552 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50acfb72-2513-468d-b3fa-da2bc13da098 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:02.878182 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:02.964442 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50acfb72-2513-468d-b3fa-da2bc13da098 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:03.000825 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5834a075-4d47-447b-aaea-d981da57152a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:03.001612 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5834a075-4d47-447b-aaea-d981da57152a None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:03.002104 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5834a075-4d47-447b-aaea-d981da57152a None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:03.002382 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5834a075-4d47-447b-aaea-d981da57152a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:03.012469 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5834a075-4d47-447b-aaea-d981da57152a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:35:03.019932 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-5834a075-4d47-447b-aaea-d981da57152a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:35:03.019932 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:03.019932 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:03.019932 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:03.019932 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:03.019932 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:03.019932 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:03.019932 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:03.019932 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:03.019932 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:03.019932 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:03.019932 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:03.019932 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:35:03.019932 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:35:03.019932 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:35:03.019932 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:35:03.021088 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:35:03.021088 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:35:03.021088 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:35:03.021088 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:35:03.021088 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:03.021088 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:03.021088 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:03.021088 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:03.021088 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:03.021088 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:03.021088 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:03.021088 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:03.021088 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:03.021088 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:03.021088 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:03.021088 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:03.022127 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:03.022127 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:03.022127 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:03.022127 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:03.022127 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:35:03.022127 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:35:03.022127 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:35:03.022127 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:35:03.022127 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:35:03.022127 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:35:03.022127 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:35:03.022127 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:35:03.022127 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 635/1263] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jun 15 14:35:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:03.027870 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff839717-6a7c-4bb8-8244-70fcc2e57344 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:03.046373 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:03.126119 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff839717-6a7c-4bb8-8244-70fcc2e57344 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:03.160156 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f06793-b195-46b3-81e6-90506227a608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:03.160824 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f06793-b195-46b3-81e6-90506227a608 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:03.161155 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f06793-b195-46b3-81e6-90506227a608 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:03.161487 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f06793-b195-46b3-81e6-90506227a608 None admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:03.162258 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f06793-b195-46b3-81e6-90506227a608 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:03.162566 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f06793-b195-46b3-81e6-90506227a608 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:03.163921 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f06793-b195-46b3-81e6-90506227a608 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:03.172460 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 629/1264] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jun 15 14:35:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:35:03.178532 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5834a075-4d47-447b-aaea-d981da57152a None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:03.244364 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:03.276212 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5834a075-4d47-447b-aaea-d981da57152a None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:03.309355 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29025a5c-ed47-4dd9-947b-63581f3d0b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:03.310132 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29025a5c-ed47-4dd9-947b-63581f3d0b43 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:03.310415 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29025a5c-ed47-4dd9-947b-63581f3d0b43 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:03.310688 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29025a5c-ed47-4dd9-947b-63581f3d0b43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:03.320859 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29025a5c-ed47-4dd9-947b-63581f3d0b43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:03.322292 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29025a5c-ed47-4dd9-947b-63581f3d0b43 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:03.339718 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-29025a5c-ed47-4dd9-947b-63581f3d0b43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:03.339718 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:03.339718 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:03.339718 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:03.339718 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:03.339718 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:03.339718 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:03.339718 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:03.339718 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:03.339718 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:03.339718 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:03.339718 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:03.339718 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:35:03.339718 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:35:03.339718 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:35:03.340872 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:35:03.340872 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:03.340872 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:03.340872 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:03.340872 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:03.340872 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:03.340872 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:03.340872 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:03.340872 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:03.340872 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:03.340872 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:03.340872 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:03.340872 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:03.340872 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:03.340872 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:03.340872 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:03.340872 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:03.341758 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:35:03.341758 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:35:03.341758 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:03.341758 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:35:03.341758 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 636/1265] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:35:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:03.346860 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f06793-b195-46b3-81e6-90506227a608 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:03.426780 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:03.445847 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f06793-b195-46b3-81e6-90506227a608 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:03.482787 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42318113-46bb-4645-9268-705e95a45c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:03.483446 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42318113-46bb-4645-9268-705e95a45c37 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:03.483724 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42318113-46bb-4645-9268-705e95a45c37 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:03.483998 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42318113-46bb-4645-9268-705e95a45c37 None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:03.484761 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42318113-46bb-4645-9268-705e95a45c37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:03.485045 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42318113-46bb-4645-9268-705e95a45c37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:03.486395 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42318113-46bb-4645-9268-705e95a45c37 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:03.498877 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 630/1266] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:35:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:03.504728 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29025a5c-ed47-4dd9-947b-63581f3d0b43 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:03.606846 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29025a5c-ed47-4dd9-947b-63581f3d0b43 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:03.626939 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:03.641311 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33c770a-6446-4df1-a770-d9bf4bd06fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:03.642107 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33c770a-6446-4df1-a770-d9bf4bd06fe7 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:03.642404 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33c770a-6446-4df1-a770-d9bf4bd06fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:03.642709 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33c770a-6446-4df1-a770-d9bf4bd06fe7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:03.650390 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33c770a-6446-4df1-a770-d9bf4bd06fe7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:03.651920 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33c770a-6446-4df1-a770-d9bf4bd06fe7 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:03.665982 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-c33c770a-6446-4df1-a770-d9bf4bd06fe7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 15 14:35:03.665982 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:03.665982 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:03.665982 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:03.665982 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:03.665982 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:03.665982 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:03.665982 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:03.665982 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:03.665982 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:03.665982 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:03.665982 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:03.665982 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:35:03.665982 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:35:03.665982 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:35:03.665982 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:35:03.667093 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:03.667093 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:03.667093 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:35:03.667093 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:35:03.667093 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:35:03.667093 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:35:03.667093 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:03.667093 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:03.667093 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:03.667093 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:03.667093 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:03.667093 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:03.667093 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:03.667093 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:03.667093 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:03.667093 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:03.667093 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:03.667093 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:03.668670 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:03.668670 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:03.668670 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:35:03.668670 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:35:03.668670 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:35:03.668670 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:35:03.668670 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:35:03.668670 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:35:03.668670 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 15 14:35:03.668670 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:35:03.668670 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 637/1267] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:35:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:03.673607 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42318113-46bb-4645-9268-705e95a45c37 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:03.773567 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42318113-46bb-4645-9268-705e95a45c37 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:03.809053 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15516add-9870-44ad-b36c-cfda525d8fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:03.809875 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15516add-9870-44ad-b36c-cfda525d8fe4 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:03.810153 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15516add-9870-44ad-b36c-cfda525d8fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:03.810432 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15516add-9870-44ad-b36c-cfda525d8fe4 None admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:03.811257 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15516add-9870-44ad-b36c-cfda525d8fe4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:03.811566 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15516add-9870-44ad-b36c-cfda525d8fe4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:03.813019 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15516add-9870-44ad-b36c-cfda525d8fe4 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:03.821597 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:03.829801 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 631/1268] 10.222.0.42 () {56 vars in 1143 bytes} [Wed Jun 15 14:35:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:03.835616 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33c770a-6446-4df1-a770-d9bf4bd06fe7 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:03.936680 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33c770a-6446-4df1-a770-d9bf4bd06fe7 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:03.970367 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503d0ba4-963c-42d1-9b77-0f5d88440c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:03.971203 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d0ba4-963c-42d1-9b77-0f5d88440c67 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:03.971491 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d0ba4-963c-42d1-9b77-0f5d88440c67 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:03.971770 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d0ba4-963c-42d1-9b77-0f5d88440c67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:03.984062 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503d0ba4-963c-42d1-9b77-0f5d88440c67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:03.987365 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503d0ba4-963c-42d1-9b77-0f5d88440c67 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:04.002869 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-503d0ba4-963c-42d1-9b77-0f5d88440c67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:04.002869 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:04.002869 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:04.002869 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:04.002869 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:04.002869 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:04.002869 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:04.002869 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:04.002869 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:04.002869 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:04.002869 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:04.002869 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:04.002869 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:35:04.002869 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:35:04.002869 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:35:04.003922 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:35:04.003922 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:04.003922 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:04.003922 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:04.003922 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:04.003922 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:04.003922 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:04.003922 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:04.003922 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:04.003922 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:04.003922 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:04.003922 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:04.003922 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:04.003922 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:04.003922 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:04.003922 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:04.003922 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:04.005096 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:35:04.005096 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:35:04.005096 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:04.005096 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:35:04.005096 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 638/1269] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:35:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:04.009880 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15516add-9870-44ad-b36c-cfda525d8fe4 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:04.113763 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15516add-9870-44ad-b36c-cfda525d8fe4 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:04.146074 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04c90d5-dddc-47bd-bea5-60d4e719558e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:04.146725 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04c90d5-dddc-47bd-bea5-60d4e719558e None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:04.147008 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04c90d5-dddc-47bd-bea5-60d4e719558e None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:04.147285 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04c90d5-dddc-47bd-bea5-60d4e719558e None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:04.147997 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04c90d5-dddc-47bd-bea5-60d4e719558e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:04.148583 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04c90d5-dddc-47bd-bea5-60d4e719558e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:04.149938 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04c90d5-dddc-47bd-bea5-60d4e719558e None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:04.161545 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 632/1270] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:35:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:04.167213 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503d0ba4-963c-42d1-9b77-0f5d88440c67 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:04.199960 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:04.266059 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503d0ba4-963c-42d1-9b77-0f5d88440c67 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:04.299936 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4e03ec-db2c-4af8-81fe-1c9415d53bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:04.300824 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e03ec-db2c-4af8-81fe-1c9415d53bd0 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:04.301366 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e03ec-db2c-4af8-81fe-1c9415d53bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:04.301937 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e03ec-db2c-4af8-81fe-1c9415d53bd0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:04.309319 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4e03ec-db2c-4af8-81fe-1c9415d53bd0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:04.310744 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4e03ec-db2c-4af8-81fe-1c9415d53bd0 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:04.325429 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-8f4e03ec-db2c-4af8-81fe-1c9415d53bd0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:35:04.325429 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:04.325429 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:04.325429 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:04.325429 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:04.325429 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:04.325429 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:04.325429 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:04.325429 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:04.325429 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:04.325429 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:04.325429 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:04.325429 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:35:04.325429 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:35:04.325429 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:35:04.326476 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:35:04.326476 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:04.326476 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:04.326476 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:04.326476 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:04.326476 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:04.326476 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:04.326476 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:04.326476 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:04.326476 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:04.326476 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:04.326476 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:04.326476 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:04.326476 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:04.326476 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:04.326476 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:04.326476 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:04.327590 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:35:04.327590 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:35:04.327590 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:35:04.327590 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:35:04.327590 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:35:04.327590 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:35:04.327590 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:35:04.327590 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:35:04.327590 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 639/1271] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jun 15 14:35:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:04.332413 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04c90d5-dddc-47bd-bea5-60d4e719558e None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:04.390014 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:04.431166 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04c90d5-dddc-47bd-bea5-60d4e719558e None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:04.463480 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036e54ee-8939-4900-8bd5-562379a4470b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:04.464157 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e54ee-8939-4900-8bd5-562379a4470b None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:04.465194 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e54ee-8939-4900-8bd5-562379a4470b None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:04.465696 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e54ee-8939-4900-8bd5-562379a4470b None admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:04.466492 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036e54ee-8939-4900-8bd5-562379a4470b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:04.466808 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036e54ee-8939-4900-8bd5-562379a4470b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:04.468255 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036e54ee-8939-4900-8bd5-562379a4470b None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:04.480269 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 633/1272] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jun 15 14:35:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:04.486335 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4e03ec-db2c-4af8-81fe-1c9415d53bd0 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:04.578808 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:04.587216 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4e03ec-db2c-4af8-81fe-1c9415d53bd0 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:04.621200 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5d77d7-da27-451d-9bd8-b45a6fbb319e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:04.622082 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d77d7-da27-451d-9bd8-b45a6fbb319e None admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:04.622360 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d77d7-da27-451d-9bd8-b45a6fbb319e None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:04.622636 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d77d7-da27-451d-9bd8-b45a6fbb319e None admin] PATH_INFO: `/v3/projects/3094ff8966f149bda784da6a51b1358f/users/2bb6671c19464424a6e9cd4519ac93e2/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:04.624616 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5d77d7-da27-451d-9bd8-b45a6fbb319e None admin] RBAC: Authorizing `identity:create_grant(project_id=3094ff8966f149bda784da6a51b1358f, user_id=2bb6671c19464424a6e9cd4519ac93e2, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:04.626037 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5d77d7-da27-451d-9bd8-b45a6fbb319e None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:04.722987 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 640/1273] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Jun 15 14:35:04 2022] PUT /identity/v3/projects/3094ff8966f149bda784da6a51b1358f/users/2bb6671c19464424a6e9cd4519ac93e2/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:35:06.825694 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0179b6-15f2-4510-a4e3-8d379426760b None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:06.826232 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0179b6-15f2-4510-a4e3-8d379426760b None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:06.826232 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0179b6-15f2-4510-a4e3-8d379426760b None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:06.827617 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 634/1274] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:35:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:06.832793 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d6699-d7d0-47d9-b4c1-5429c26f22f6 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:06.833283 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d6699-d7d0-47d9-b4c1-5429c26f22f6 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:06.833377 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d6699-d7d0-47d9-b4c1-5429c26f22f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:06.849892 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-ba9d6699-d7d0-47d9-b4c1-5429c26f22f6 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:06.851073 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-ba9d6699-d7d0-47d9-b4c1-5429c26f22f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:06.918224 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 641/1275] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:35:06.943054 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0179b6-15f2-4510-a4e3-8d379426760b None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:07.045344 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0179b6-15f2-4510-a4e3-8d379426760b None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:07.062987 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:07.079876 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dbf0c9-01c4-4c4e-b014-946608808091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:07.080623 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dbf0c9-01c4-4c4e-b014-946608808091 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:07.080907 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dbf0c9-01c4-4c4e-b014-946608808091 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:07.081181 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dbf0c9-01c4-4c4e-b014-946608808091 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:07.091175 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dbf0c9-01c4-4c4e-b014-946608808091 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:35:07.098757 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-35dbf0c9-01c4-4c4e-b014-946608808091 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:35:07.098757 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:07.098757 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:07.098757 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:07.098757 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:07.098757 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:07.098757 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:07.098757 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:07.098757 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:07.098757 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:07.098757 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:07.098757 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:07.098757 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:35:07.098757 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:35:07.098757 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:35:07.098757 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:35:07.099862 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:35:07.099862 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:35:07.099862 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:35:07.099862 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:35:07.099862 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:07.099862 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:07.099862 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:07.099862 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:07.099862 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:07.099862 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:07.099862 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:07.099862 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:07.099862 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:07.099862 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:07.099862 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:07.099862 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:07.101115 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:07.101115 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:07.101115 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:07.101115 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:07.101115 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:35:07.101115 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:35:07.101115 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:35:07.101115 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:35:07.101115 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:35:07.101115 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:35:07.101115 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:35:07.101115 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:35:07.101115 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 635/1276] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jun 15 14:35:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:07.106299 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9d6699-d7d0-47d9-b4c1-5429c26f22f6 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:07.188008 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:07.205905 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9d6699-d7d0-47d9-b4c1-5429c26f22f6 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:07.237902 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62be2794-14b8-4e2c-a91b-d31fe42a8e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:07.238554 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62be2794-14b8-4e2c-a91b-d31fe42a8e96 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:07.238839 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62be2794-14b8-4e2c-a91b-d31fe42a8e96 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:07.239115 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62be2794-14b8-4e2c-a91b-d31fe42a8e96 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:07.239855 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62be2794-14b8-4e2c-a91b-d31fe42a8e96 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:07.240188 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62be2794-14b8-4e2c-a91b-d31fe42a8e96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:07.241541 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62be2794-14b8-4e2c-a91b-d31fe42a8e96 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:07.250095 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 642/1277] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jun 15 14:35:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:07.255802 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dbf0c9-01c4-4c4e-b014-946608808091 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:07.353756 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dbf0c9-01c4-4c4e-b014-946608808091 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:07.388415 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e3e342-6d79-4cc0-8973-d2ae956f1d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:07.389201 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e3e342-6d79-4cc0-8973-d2ae956f1d53 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:07.394147 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e3e342-6d79-4cc0-8973-d2ae956f1d53 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:07.394147 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e3e342-6d79-4cc0-8973-d2ae956f1d53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:07.400473 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e3e342-6d79-4cc0-8973-d2ae956f1d53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:07.402097 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e3e342-6d79-4cc0-8973-d2ae956f1d53 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:07.419759 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-75e3e342-6d79-4cc0-8973-d2ae956f1d53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:07.419759 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:07.419759 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:07.419759 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:07.419759 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:07.419759 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:07.419759 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:07.419759 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:07.419759 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:07.419759 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:07.419759 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:07.419759 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:07.419759 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:35:07.419759 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:35:07.419759 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:35:07.425130 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:35:07.425130 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:07.425130 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:07.425130 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:07.425130 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:07.425130 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:07.425130 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:07.425130 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:07.425130 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:07.425130 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:07.425130 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:07.425130 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:07.425130 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:07.425130 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:07.425130 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:07.425130 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:07.425130 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:07.426384 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:35:07.426384 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:35:07.426384 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:07.426384 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:35:07.426384 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 636/1278] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:35:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:07.426981 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62be2794-14b8-4e2c-a91b-d31fe42a8e96 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:07.528778 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62be2794-14b8-4e2c-a91b-d31fe42a8e96 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:07.564500 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77441469-a23b-4e8f-ad4f-9c017c60a19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:07.565167 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77441469-a23b-4e8f-ad4f-9c017c60a19c None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:07.565453 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77441469-a23b-4e8f-ad4f-9c017c60a19c None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:07.565734 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77441469-a23b-4e8f-ad4f-9c017c60a19c None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:07.566457 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77441469-a23b-4e8f-ad4f-9c017c60a19c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:07.566769 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77441469-a23b-4e8f-ad4f-9c017c60a19c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:07.568153 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77441469-a23b-4e8f-ad4f-9c017c60a19c None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:07.579737 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 643/1279] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:35:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:35:07.585794 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e3e342-6d79-4cc0-8973-d2ae956f1d53 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:07.616921 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:07.690769 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e3e342-6d79-4cc0-8973-d2ae956f1d53 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:07.723943 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75629fad-af33-4c44-82ff-52b46ee809b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:07.724766 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75629fad-af33-4c44-82ff-52b46ee809b6 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:07.725052 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75629fad-af33-4c44-82ff-52b46ee809b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:07.725331 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75629fad-af33-4c44-82ff-52b46ee809b6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:07.734087 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75629fad-af33-4c44-82ff-52b46ee809b6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:07.735517 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75629fad-af33-4c44-82ff-52b46ee809b6 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:07.749768 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-75629fad-af33-4c44-82ff-52b46ee809b6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 15 14:35:07.749768 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:07.749768 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:07.749768 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:07.749768 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:07.749768 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:07.749768 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:07.749768 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:07.749768 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:07.749768 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:07.749768 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:07.749768 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:07.749768 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:35:07.749768 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:35:07.749768 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:35:07.749768 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:35:07.750929 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:07.750929 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:07.750929 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:35:07.750929 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:35:07.750929 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:35:07.750929 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:35:07.750929 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:07.750929 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:07.750929 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:07.750929 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:07.750929 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:07.750929 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:07.750929 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:07.750929 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:07.750929 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:07.750929 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:07.750929 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:07.750929 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:07.756233 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:07.756233 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:07.756233 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:35:07.756233 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:35:07.756233 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:35:07.756233 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:35:07.756233 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:35:07.756233 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:35:07.756233 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 15 14:35:07.756233 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:35:07.756233 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 637/1280] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:35:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:07.757074 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77441469-a23b-4e8f-ad4f-9c017c60a19c None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:07.760864 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:07.853297 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77441469-a23b-4e8f-ad4f-9c017c60a19c None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:07.886401 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9956980d-e206-4400-9ed0-f5870701cd38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:07.887162 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956980d-e206-4400-9ed0-f5870701cd38 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:07.887446 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956980d-e206-4400-9ed0-f5870701cd38 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:07.887727 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956980d-e206-4400-9ed0-f5870701cd38 None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:07.888633 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9956980d-e206-4400-9ed0-f5870701cd38 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:07.888942 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9956980d-e206-4400-9ed0-f5870701cd38 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:07.890872 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9956980d-e206-4400-9ed0-f5870701cd38 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:07.909525 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 644/1281] 10.222.0.42 () {56 vars in 1143 bytes} [Wed Jun 15 14:35:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:35:07.915268 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75629fad-af33-4c44-82ff-52b46ee809b6 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:08.007668 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:08.017054 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75629fad-af33-4c44-82ff-52b46ee809b6 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:08.049928 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c06f5f9-b5de-4c7f-9979-90ce39aa4e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:08.050570 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06f5f9-b5de-4c7f-9979-90ce39aa4e1b None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:08.050853 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06f5f9-b5de-4c7f-9979-90ce39aa4e1b None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:08.051130 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06f5f9-b5de-4c7f-9979-90ce39aa4e1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:08.115820 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c06f5f9-b5de-4c7f-9979-90ce39aa4e1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:08.117293 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c06f5f9-b5de-4c7f-9979-90ce39aa4e1b None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:08.204942 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:08.216177 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 638/1282] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jun 15 14:35:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:35:08.222085 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9956980d-e206-4400-9ed0-f5870701cd38 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:08.282284 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:08.320379 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9956980d-e206-4400-9ed0-f5870701cd38 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:08.358184 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b33920-c563-49f6-859c-a761a182bc9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:08.358986 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b33920-c563-49f6-859c-a761a182bc9e None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:08.359269 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b33920-c563-49f6-859c-a761a182bc9e None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:08.359544 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b33920-c563-49f6-859c-a761a182bc9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:08.370800 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b33920-c563-49f6-859c-a761a182bc9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:08.372289 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b33920-c563-49f6-859c-a761a182bc9e None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:08.392555 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-08b33920-c563-49f6-859c-a761a182bc9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:08.392555 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:08.392555 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:08.392555 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:08.392555 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:08.392555 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:08.392555 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:08.392555 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:08.392555 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:08.392555 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:08.392555 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:08.392555 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:08.392555 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:35:08.392555 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:35:08.392555 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:35:08.394378 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:35:08.394378 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:08.394378 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:08.394378 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:08.394378 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:08.394378 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:08.394378 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:08.394378 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:08.394378 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:08.394378 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:08.394378 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:08.394378 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:08.394378 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:08.394378 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:08.394378 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:08.394378 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:08.394378 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:08.395417 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:35:08.395417 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:35:08.395417 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:08.395417 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:35:08.395417 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 645/1283] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:35:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:08.399605 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c06f5f9-b5de-4c7f-9979-90ce39aa4e1b None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:08.510605 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c06f5f9-b5de-4c7f-9979-90ce39aa4e1b None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:08.551783 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cf0030-1633-4099-87ce-e07437b7a53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:08.552499 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf0030-1633-4099-87ce-e07437b7a53f None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:08.552783 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf0030-1633-4099-87ce-e07437b7a53f None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:08.553059 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf0030-1633-4099-87ce-e07437b7a53f None admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:08.553792 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cf0030-1633-4099-87ce-e07437b7a53f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:08.554103 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cf0030-1633-4099-87ce-e07437b7a53f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:08.555511 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cf0030-1633-4099-87ce-e07437b7a53f None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:08.574767 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 639/1284] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jun 15 14:35:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:08.575153 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b33920-c563-49f6-859c-a761a182bc9e None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:08.682279 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b33920-c563-49f6-859c-a761a182bc9e None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:08.696976 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:08.717880 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a1225d-7d8d-4626-983f-815ac5ba66be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:08.718671 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a1225d-7d8d-4626-983f-815ac5ba66be None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:08.718946 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a1225d-7d8d-4626-983f-815ac5ba66be None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:08.719213 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a1225d-7d8d-4626-983f-815ac5ba66be None admin] PATH_INFO: `/v3/projects/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:08.726118 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a1225d-7d8d-4626-983f-815ac5ba66be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:08.727607 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a1225d-7d8d-4626-983f-815ac5ba66be None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:08.742231 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-55a1225d-7d8d-4626-983f-815ac5ba66be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:35:08.742231 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:08.742231 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:08.742231 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:08.742231 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:08.742231 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:08.742231 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:08.742231 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:08.742231 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:08.742231 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:08.742231 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:08.742231 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:08.742231 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:35:08.742231 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:35:08.742231 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:35:08.743418 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:35:08.743418 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:08.743418 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:08.743418 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:08.743418 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:08.743418 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:08.743418 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:08.743418 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:08.743418 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:08.743418 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:08.743418 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:08.743418 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:08.743418 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:08.743418 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:08.743418 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:08.743418 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:08.743418 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:08.744542 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:35:08.744542 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:35:08.744542 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:35:08.744542 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:35:08.744542 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:35:08.744542 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:35:08.744542 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:35:08.744542 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 15 14:35:08.744542 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 646/1285] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jun 15 14:35:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:08.750744 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cf0030-1633-4099-87ce-e07437b7a53f None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:08.805497 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:08.854841 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cf0030-1633-4099-87ce-e07437b7a53f None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:08.887823 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d792884-ff93-4112-859d-b519344713d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:08.888561 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d792884-ff93-4112-859d-b519344713d1 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:08.888868 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d792884-ff93-4112-859d-b519344713d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:08.889150 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d792884-ff93-4112-859d-b519344713d1 None admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:08.889932 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d792884-ff93-4112-859d-b519344713d1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:08.890249 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d792884-ff93-4112-859d-b519344713d1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:08.891677 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d792884-ff93-4112-859d-b519344713d1 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:08.904704 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 640/1286] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jun 15 14:35:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:08.910798 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a1225d-7d8d-4626-983f-815ac5ba66be None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:09.008727 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a1225d-7d8d-4626-983f-815ac5ba66be None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:09.041291 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ec7976-e43d-4980-9657-8d9640cd9db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:09.041931 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ec7976-e43d-4980-9657-8d9640cd9db9 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:09.042217 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ec7976-e43d-4980-9657-8d9640cd9db9 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:09.042992 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ec7976-e43d-4980-9657-8d9640cd9db9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:09.043877 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ec7976-e43d-4980-9657-8d9640cd9db9 None admin] RBAC: Adding query filter params (role.id=2bd1979f53bb4853a50dd5bee6aa0782, scope.project.id=3094ff8966f149bda784da6a51b1358f, user.id=2bb6671c19464424a6e9cd4519ac93e2) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:09.044217 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ec7976-e43d-4980-9657-8d9640cd9db9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:09.045649 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ec7976-e43d-4980-9657-8d9640cd9db9 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:09.051591 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 647/1287] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jun 15 14:35:08 2022] GET /identity/v3/role_assignments?user.id=2bb6671c19464424a6e9cd4519ac93e2&scope.project.id=3094ff8966f149bda784da6a51b1358f&role.id=2bd1979f53bb4853a50dd5bee6aa0782 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:11.185036 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a906d724-cfc8-4b66-b788-a88de9775823 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:11.185680 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a906d724-cfc8-4b66-b788-a88de9775823 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:11.185680 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a906d724-cfc8-4b66-b788-a88de9775823 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:11.186938 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 641/1288] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:35:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:11.192009 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa93fe66-7f53-44b1-8932-18ceba835ecf None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:11.192439 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa93fe66-7f53-44b1-8932-18ceba835ecf None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:11.192588 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa93fe66-7f53-44b1-8932-18ceba835ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:11.209456 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-fa93fe66-7f53-44b1-8932-18ceba835ecf None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:11.210614 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-fa93fe66-7f53-44b1-8932-18ceba835ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:11.239439 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:11.277355 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 648/1289] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:35:11.299845 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a906d724-cfc8-4b66-b788-a88de9775823 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:11.398624 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a906d724-cfc8-4b66-b788-a88de9775823 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:11.410675 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:11.433825 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f476ac47-5420-4684-8e11-404efe809a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:11.434616 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f476ac47-5420-4684-8e11-404efe809a57 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:11.434899 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f476ac47-5420-4684-8e11-404efe809a57 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:11.435228 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f476ac47-5420-4684-8e11-404efe809a57 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:11.435820 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f476ac47-5420-4684-8e11-404efe809a57 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:11.437204 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f476ac47-5420-4684-8e11-404efe809a57 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:11.448807 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-f476ac47-5420-4684-8e11-404efe809a57 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 14:35:11.448807 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:11.448807 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:35:11.448807 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:35:11.448807 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:35:11.448807 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:35:11.448807 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:35:11.448807 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:35:11.448807 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 14:35:11.448807 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:35:11.448807 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:35:11.448807 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:35:11.448807 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:11.448807 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:11.448807 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:11.448807 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:11.448807 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:11.449917 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:11.449917 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:11.449917 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:11.449917 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:11.449917 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:11.449917 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:11.449917 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:35:11.449917 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:35:11.449917 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:35:11.449917 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:35:11.449917 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:11.449917 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:11.449917 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:11.449917 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:11.449917 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:11.449917 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:11.450845 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:11.450845 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:11.450845 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:11.450845 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:11.450845 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:11.450845 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:11.450845 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:11.450845 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:11.450845 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:11.450845 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:11.450845 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:35:11.450845 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:35:11.450845 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 14:35:11.450845 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:35:11.450845 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 642/1290] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Jun 15 14:35:11 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:11.455983 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa93fe66-7f53-44b1-8932-18ceba835ecf None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:11.554154 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa93fe66-7f53-44b1-8932-18ceba835ecf None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:11.587269 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212680f8-87a5-441b-9a69-5d5c4cd4de99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:11.587926 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212680f8-87a5-441b-9a69-5d5c4cd4de99 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:11.588242 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212680f8-87a5-441b-9a69-5d5c4cd4de99 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:11.588522 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212680f8-87a5-441b-9a69-5d5c4cd4de99 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:11.589733 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212680f8-87a5-441b-9a69-5d5c4cd4de99 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:11.590161 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212680f8-87a5-441b-9a69-5d5c4cd4de99 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:11.591514 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212680f8-87a5-441b-9a69-5d5c4cd4de99 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:11.596311 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 649/1291] 10.222.0.42 () {56 vars in 1116 bytes} [Wed Jun 15 14:35:11 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:11.602275 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f476ac47-5420-4684-8e11-404efe809a57 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:11.700770 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f476ac47-5420-4684-8e11-404efe809a57 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:11.734303 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3911ba57-a3ed-4b1e-9981-1800c760ed94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:11.734956 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3911ba57-a3ed-4b1e-9981-1800c760ed94 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:11.735236 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3911ba57-a3ed-4b1e-9981-1800c760ed94 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:11.735508 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3911ba57-a3ed-4b1e-9981-1800c760ed94 None admin] PATH_INFO: `/v3/services` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:11.736231 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3911ba57-a3ed-4b1e-9981-1800c760ed94 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:11.736538 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3911ba57-a3ed-4b1e-9981-1800c760ed94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:11.737890 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3911ba57-a3ed-4b1e-9981-1800c760ed94 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:11.742366 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 643/1292] 10.222.0.42 () {56 vars in 1116 bytes} [Wed Jun 15 14:35:11 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:13.882631 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dfa957-855c-4448-aba1-0bb564e9affc None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:13.883129 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dfa957-855c-4448-aba1-0bb564e9affc None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:13.883129 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dfa957-855c-4448-aba1-0bb564e9affc None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:13.884537 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 650/1293] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:35:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:13.890684 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a1d31-e152-4ec9-b768-7b947cd69a3c None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:13.891105 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a1d31-e152-4ec9-b768-7b947cd69a3c None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:13.891243 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a1d31-e152-4ec9-b768-7b947cd69a3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:13.907777 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-048a1d31-e152-4ec9-b768-7b947cd69a3c None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:13.908980 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-048a1d31-e152-4ec9-b768-7b947cd69a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:13.937419 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:13.972726 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 644/1294] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:35:13.997040 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dfa957-855c-4448-aba1-0bb564e9affc None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:14.030094 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:14.099573 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dfa957-855c-4448-aba1-0bb564e9affc None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:14.134819 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb817d71-06ce-4f4e-a16d-cbc398962c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:14.135409 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb817d71-06ce-4f4e-a16d-cbc398962c0e None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:14.135699 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb817d71-06ce-4f4e-a16d-cbc398962c0e None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:14.136043 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb817d71-06ce-4f4e-a16d-cbc398962c0e None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:14.136799 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb817d71-06ce-4f4e-a16d-cbc398962c0e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:14.138163 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb817d71-06ce-4f4e-a16d-cbc398962c0e None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:14.238624 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 651/1295] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Jun 15 14:35:13 2022] POST /identity/v3/services => generated 246 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:35:16.365394 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6edc4ab-1dc9-444b-a02b-1970dbaaf667 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:16.365964 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6edc4ab-1dc9-444b-a02b-1970dbaaf667 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:16.365964 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6edc4ab-1dc9-444b-a02b-1970dbaaf667 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:16.367342 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 645/1296] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:35:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:16.372591 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9345e679-13f1-4b7a-b72f-bb695f2ac79f None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:16.373013 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9345e679-13f1-4b7a-b72f-bb695f2ac79f None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:16.373200 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9345e679-13f1-4b7a-b72f-bb695f2ac79f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:16.390347 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-9345e679-13f1-4b7a-b72f-bb695f2ac79f None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:16.391609 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-9345e679-13f1-4b7a-b72f-bb695f2ac79f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:16.466609 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 652/1297] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:16 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:35:16.488913 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6edc4ab-1dc9-444b-a02b-1970dbaaf667 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:16.590156 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6edc4ab-1dc9-444b-a02b-1970dbaaf667 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:16.623288 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42edf98-71cc-44b5-952d-38bb880d23d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:16.624018 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42edf98-71cc-44b5-952d-38bb880d23d5 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:16.624330 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42edf98-71cc-44b5-952d-38bb880d23d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:16.624607 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42edf98-71cc-44b5-952d-38bb880d23d5 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:16.625249 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42edf98-71cc-44b5-952d-38bb880d23d5 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:16.626603 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42edf98-71cc-44b5-952d-38bb880d23d5 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:16.638787 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-e42edf98-71cc-44b5-952d-38bb880d23d5 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 14:35:16.638787 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:16.638787 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:35:16.638787 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:35:16.638787 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:35:16.638787 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:35:16.638787 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:35:16.638787 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:35:16.638787 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 14:35:16.638787 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:35:16.638787 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:35:16.638787 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:35:16.638787 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:16.638787 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:16.638787 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:16.638787 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:16.638787 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:16.639976 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:16.639976 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:16.639976 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:16.639976 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:16.639976 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:16.639976 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:16.639976 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:35:16.639976 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:35:16.639976 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:35:16.639976 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:35:16.639976 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:16.639976 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:16.639976 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:16.639976 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:16.639976 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:16.639976 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:16.640857 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:16.640857 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:16.640857 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:16.640857 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:16.640857 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:16.640857 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:16.640857 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:16.640857 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:16.640857 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:16.640857 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:16.640857 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:35:16.640857 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:35:16.640857 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 14:35:16.640857 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:35:16.640857 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 646/1298] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Jun 15 14:35:16 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:16.646230 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9345e679-13f1-4b7a-b72f-bb695f2ac79f None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:16.665712 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:16.743419 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9345e679-13f1-4b7a-b72f-bb695f2ac79f None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:16.778494 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969bf103-3e47-463e-8010-8eecb85d22b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:16.779159 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969bf103-3e47-463e-8010-8eecb85d22b0 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:16.779432 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969bf103-3e47-463e-8010-8eecb85d22b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:16.779704 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969bf103-3e47-463e-8010-8eecb85d22b0 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:16.780433 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969bf103-3e47-463e-8010-8eecb85d22b0 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:16.780739 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969bf103-3e47-463e-8010-8eecb85d22b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:16.782085 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969bf103-3e47-463e-8010-8eecb85d22b0 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:16.787043 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 653/1299] 10.222.0.42 () {56 vars in 1116 bytes} [Wed Jun 15 14:35:16 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:16.793254 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42edf98-71cc-44b5-952d-38bb880d23d5 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:16.825312 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:16.891349 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42edf98-71cc-44b5-952d-38bb880d23d5 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:16.924492 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b40fe4-fe60-46a6-9ca6-0ead745b6e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:16.925129 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b40fe4-fe60-46a6-9ca6-0ead745b6e56 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:16.925409 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b40fe4-fe60-46a6-9ca6-0ead745b6e56 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:16.925680 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b40fe4-fe60-46a6-9ca6-0ead745b6e56 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:16.926446 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b40fe4-fe60-46a6-9ca6-0ead745b6e56 None admin] RBAC: Adding query filter params (interface=public, service_id=d732efbdd29c4d969b62d4e6cf392482, region_id=RegionOne) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:16.926754 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b40fe4-fe60-46a6-9ca6-0ead745b6e56 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:16.928158 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b40fe4-fe60-46a6-9ca6-0ead745b6e56 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:16.933539 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 647/1300] 10.222.0.42 () {56 vars in 1251 bytes} [Wed Jun 15 14:35:16 2022] GET /identity/v3/endpoints?service_id=d732efbdd29c4d969b62d4e6cf392482&interface=public®ion_id=RegionOne => generated 194 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:16.939616 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969bf103-3e47-463e-8010-8eecb85d22b0 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:17.022862 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:17.040412 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969bf103-3e47-463e-8010-8eecb85d22b0 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:17.074171 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d6115-37a2-453c-ac39-efc3a4781b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:17.074929 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d6115-37a2-453c-ac39-efc3a4781b31 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:17.075216 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d6115-37a2-453c-ac39-efc3a4781b31 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:17.075494 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d6115-37a2-453c-ac39-efc3a4781b31 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:17.076194 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d6115-37a2-453c-ac39-efc3a4781b31 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:17.077730 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d6115-37a2-453c-ac39-efc3a4781b31 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:17.084536 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 654/1301] 10.222.0.42 () {56 vars in 1087 bytes} [Wed Jun 15 14:35:16 2022] GET /identity/v3/services => generated 2191 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:35:19.196023 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1654e61-5f4c-41bc-a36f-7fd9b3ff7af8 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:19.196637 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1654e61-5f4c-41bc-a36f-7fd9b3ff7af8 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:19.196637 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1654e61-5f4c-41bc-a36f-7fd9b3ff7af8 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:19.198010 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 648/1302] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:35:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:19.203268 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c943bc-c474-435e-b427-1e0ab265ac74 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:19.203742 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c943bc-c474-435e-b427-1e0ab265ac74 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:19.208797 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c943bc-c474-435e-b427-1e0ab265ac74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:19.220976 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-26c943bc-c474-435e-b427-1e0ab265ac74 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:19.222165 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-26c943bc-c474-435e-b427-1e0ab265ac74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:19.290069 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 655/1303] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:19 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:35:19.313907 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1654e61-5f4c-41bc-a36f-7fd9b3ff7af8 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:19.409454 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:19.415997 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1654e61-5f4c-41bc-a36f-7fd9b3ff7af8 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:19.450005 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb6816-4d23-4342-b4d0-b3718c5969f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:19.450784 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb6816-4d23-4342-b4d0-b3718c5969f3 None admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:19.451298 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb6816-4d23-4342-b4d0-b3718c5969f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:19.451708 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb6816-4d23-4342-b4d0-b3718c5969f3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:19.452735 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fb6816-4d23-4342-b4d0-b3718c5969f3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:19.454239 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fb6816-4d23-4342-b4d0-b3718c5969f3 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:19.466139 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application [None req-03fb6816-4d23-4342-b4d0-b3718c5969f3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 14:35:19.466139 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:19.466139 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:35:19.466139 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:35:19.466139 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:35:19.466139 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:35:19.466139 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:35:19.466139 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:35:19.466139 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 14:35:19.466139 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:35:19.466139 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:35:19.466139 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:35:19.466139 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:19.466139 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:19.466139 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:19.466139 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:19.466139 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:19.467410 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:19.467410 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:19.467410 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:19.467410 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:19.467410 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:19.467410 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:19.467410 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:35:19.467410 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:35:19.467410 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:35:19.467410 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:35:19.467410 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:19.467410 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:19.467410 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:19.467410 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:19.467410 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:19.467410 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:19.468658 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:19.468658 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:19.468658 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:19.468658 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:19.468658 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:19.468658 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:19.468658 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:19.468658 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:19.468658 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:19.468658 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:19.468658 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:35:19.468658 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:35:19.468658 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 14:35:19.468658 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: ERROR keystone.server.flask.application Jun 15 14:35:19.468658 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 649/1304] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Jun 15 14:35:19 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:19.473521 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c943bc-c474-435e-b427-1e0ab265ac74 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:19.551121 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:19.573921 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c943bc-c474-435e-b427-1e0ab265ac74 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:19.606186 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7777343-0a5c-4332-aa3b-607a62ab9ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:19.606863 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7777343-0a5c-4332-aa3b-607a62ab9ac2 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:19.607141 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7777343-0a5c-4332-aa3b-607a62ab9ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:19.607477 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7777343-0a5c-4332-aa3b-607a62ab9ac2 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:19.608222 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7777343-0a5c-4332-aa3b-607a62ab9ac2 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:19.608527 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7777343-0a5c-4332-aa3b-607a62ab9ac2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:19.609865 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7777343-0a5c-4332-aa3b-607a62ab9ac2 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:19.614897 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 656/1305] 10.222.0.42 () {56 vars in 1116 bytes} [Wed Jun 15 14:35:19 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:19.621455 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb6816-4d23-4342-b4d0-b3718c5969f3 None admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:19.719112 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb6816-4d23-4342-b4d0-b3718c5969f3 None admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:19.751762 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c0864-c9b8-4602-b75c-33fd3a1ba0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:19.752431 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c0864-c9b8-4602-b75c-33fd3a1ba0e2 None admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:19.752710 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c0864-c9b8-4602-b75c-33fd3a1ba0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:19.752979 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c0864-c9b8-4602-b75c-33fd3a1ba0e2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:19.753715 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3c0864-c9b8-4602-b75c-33fd3a1ba0e2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:19.755068 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3c0864-c9b8-4602-b75c-33fd3a1ba0e2 None admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:19.850133 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 650/1306] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Jun 15 14:35:19 2022] POST /identity/v3/endpoints => generated 331 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:35:29.661575 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=30 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:29.845632 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:41:23.862896 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2698d7-5918-4fb2-9bdb-9d6004e77fd1 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:41:23.863812 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2698d7-5918-4fb2-9bdb-9d6004e77fd1 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:41:23.864302 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2698d7-5918-4fb2-9bdb-9d6004e77fd1 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:41:23.871585 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c6bccc-d90c-4c74-9326-b9d7a137441e None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:41:23.872460 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c6bccc-d90c-4c74-9326-b9d7a137441e None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:41:23.872893 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c6bccc-d90c-4c74-9326-b9d7a137441e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:41:23.876688 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 657/1307] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:41:23 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:41:23.952119 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-b2c6bccc-d90c-4c74-9326-b9d7a137441e None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:41:23.953296 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-b2c6bccc-d90c-4c74-9326-b9d7a137441e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:41:24.079821 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 651/1308] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:41:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:41:24.132210 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89198121-72b6-44bd-b679-5e27d04c254e None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:41:24.132405 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89198121-72b6-44bd-b679-5e27d04c254e None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:41:24.132707 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89198121-72b6-44bd-b679-5e27d04c254e None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:41:24.134909 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 658/1309] 10.222.0.42 () {54 vars in 964 bytes} [Wed Jun 15 14:41:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:41:24.141527 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae28e44f-25bc-472b-8b64-9de6f1e2f401 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:41:24.141826 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae28e44f-25bc-472b-8b64-9de6f1e2f401 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:41:24.142109 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae28e44f-25bc-472b-8b64-9de6f1e2f401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:41:24.256637 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-ae28e44f-25bc-472b-8b64-9de6f1e2f401 None None] MFA Rules not processed for user `c739b61c8a8b453e979bda720bafebb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:41:24.257851 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-ae28e44f-25bc-472b-8b64-9de6f1e2f401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:41:24.281112 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:41:24.349097 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 652/1310] 10.222.0.42 () {58 vars in 1061 bytes} [Wed Jun 15 14:41:24 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:41:24.365202 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89198121-72b6-44bd-b679-5e27d04c254e None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:41:24.368387 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-89198121-72b6-44bd-b679-5e27d04c254e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:41:24.534944 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89198121-72b6-44bd-b679-5e27d04c254e None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:41:24.634946 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5666b9-fb20-484c-ab6f-496f25983388 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:41:24.635517 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5666b9-fb20-484c-ab6f-496f25983388 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:41:24.636022 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5666b9-fb20-484c-ab6f-496f25983388 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:41:24.636328 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5666b9-fb20-484c-ab6f-496f25983388 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:41:24.638154 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-0c5666b9-fb20-484c-ab6f-496f25983388 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:41:24.699461 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:41:24.789306 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5666b9-fb20-484c-ab6f-496f25983388 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:41:24.791947 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5666b9-fb20-484c-ab6f-496f25983388 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:41:24.818853 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 659/1311] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:41:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:41:27.333585 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ea129-16a2-417d-bc6c-5d4b6e6c9266 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:41:27.334167 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ea129-16a2-417d-bc6c-5d4b6e6c9266 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:41:27.334167 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ea129-16a2-417d-bc6c-5d4b6e6c9266 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:41:27.336120 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 653/1312] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:41:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:41:27.340974 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c745f1e0-7a61-4322-9b90-43e2afb96f02 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:41:27.341443 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c745f1e0-7a61-4322-9b90-43e2afb96f02 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:41:27.341544 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c745f1e0-7a61-4322-9b90-43e2afb96f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:41:27.437522 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-c745f1e0-7a61-4322-9b90-43e2afb96f02 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:41:27.438761 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c745f1e0-7a61-4322-9b90-43e2afb96f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:41:27.580145 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 660/1313] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:41:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:41:27.625982 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215ea129-16a2-417d-bc6c-5d4b6e6c9266 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:41:27.630693 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-215ea129-16a2-417d-bc6c-5d4b6e6c9266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:41:27.764388 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215ea129-16a2-417d-bc6c-5d4b6e6c9266 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:41:27.851879 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:41:27.854647 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945bc77b-6283-41ba-b500-946f97cbc183 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:41:27.855309 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945bc77b-6283-41ba-b500-946f97cbc183 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:41:27.855591 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945bc77b-6283-41ba-b500-946f97cbc183 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:41:27.855867 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945bc77b-6283-41ba-b500-946f97cbc183 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:41:27.857561 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-945bc77b-6283-41ba-b500-946f97cbc183 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:41:27.991884 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945bc77b-6283-41ba-b500-946f97cbc183 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:41:27.993454 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945bc77b-6283-41ba-b500-946f97cbc183 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:41:28.014843 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 654/1314] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:41:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:41:30.751234 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf4799-195a-4b42-8724-1af177aeacac None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:41:30.751792 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf4799-195a-4b42-8724-1af177aeacac None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:41:30.751792 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf4799-195a-4b42-8724-1af177aeacac None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:41:30.753794 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 661/1315] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:41:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:41:30.759082 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b2e77-97f1-4517-926d-ed0692858c35 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:41:30.759552 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b2e77-97f1-4517-926d-ed0692858c35 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:41:30.759645 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b2e77-97f1-4517-926d-ed0692858c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:41:30.824962 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-1f7b2e77-97f1-4517-926d-ed0692858c35 None None] MFA Rules not processed for user `7fa722bf1210411e9101bfd161c43eaa`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:41:30.826099 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-1f7b2e77-97f1-4517-926d-ed0692858c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:41:30.880887 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:41:30.930028 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 655/1316] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:41:30 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:41:30.970032 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdf4799-195a-4b42-8724-1af177aeacac None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:41:30.971646 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-4bdf4799-195a-4b42-8724-1af177aeacac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:41:30.981754 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:41:31.080509 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdf4799-195a-4b42-8724-1af177aeacac None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:41:31.152617 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8835f6-036a-46a4-aeb7-eeaa3850d11e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:41:31.153264 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8835f6-036a-46a4-aeb7-eeaa3850d11e service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:41:31.153551 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8835f6-036a-46a4-aeb7-eeaa3850d11e service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:41:31.153831 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8835f6-036a-46a4-aeb7-eeaa3850d11e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:41:31.155513 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-2a8835f6-036a-46a4-aeb7-eeaa3850d11e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:41:31.218807 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:41:31.267177 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8835f6-036a-46a4-aeb7-eeaa3850d11e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:41:31.268958 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8835f6-036a-46a4-aeb7-eeaa3850d11e service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:41:31.289869 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 662/1317] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:41:30 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:41:34.318033 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f2778-3b86-4207-88ea-ebf056611d9b None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:41:34.318620 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f2778-3b86-4207-88ea-ebf056611d9b None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:41:34.318620 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f2778-3b86-4207-88ea-ebf056611d9b None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:41:34.319954 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 656/1318] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:41:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:41:34.325407 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dad0fa-9e05-46dc-9a1a-2b59ea27780e None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:41:34.325829 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dad0fa-9e05-46dc-9a1a-2b59ea27780e None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:41:34.325941 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dad0fa-9e05-46dc-9a1a-2b59ea27780e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:41:34.396621 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-30dad0fa-9e05-46dc-9a1a-2b59ea27780e None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:41:34.397877 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-30dad0fa-9e05-46dc-9a1a-2b59ea27780e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:41:34.500342 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 663/1319] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:41:34 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:41:34.555600 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e2041-eb53-48f0-8fab-67951427241a None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:41:34.555894 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e2041-eb53-48f0-8fab-67951427241a None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:41:34.556206 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e2041-eb53-48f0-8fab-67951427241a None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:41:34.557573 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 657/1320] 10.222.0.42 () {54 vars in 964 bytes} [Wed Jun 15 14:41:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:41:34.563684 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a01d87-025c-4941-9b15-715ed6730e15 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:41:34.563972 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a01d87-025c-4941-9b15-715ed6730e15 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:41:34.564269 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a01d87-025c-4941-9b15-715ed6730e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:41:34.585339 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:41:34.645359 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-33a01d87-025c-4941-9b15-715ed6730e15 None None] MFA Rules not processed for user `c739b61c8a8b453e979bda720bafebb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:41:34.646581 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-33a01d87-025c-4941-9b15-715ed6730e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:41:34.727586 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 664/1321] 10.222.0.42 () {58 vars in 1061 bytes} [Wed Jun 15 14:41:34 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:41:34.738167 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667e2041-eb53-48f0-8fab-67951427241a None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:41:34.739932 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-667e2041-eb53-48f0-8fab-67951427241a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:41:34.870223 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667e2041-eb53-48f0-8fab-67951427241a None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:41:34.971223 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47bc3df-b326-4c4b-97d7-63b1a479823e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:41:34.972401 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47bc3df-b326-4c4b-97d7-63b1a479823e service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:41:34.972687 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47bc3df-b326-4c4b-97d7-63b1a479823e service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:41:34.972972 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47bc3df-b326-4c4b-97d7-63b1a479823e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:41:34.975765 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-b47bc3df-b326-4c4b-97d7-63b1a479823e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:41:34.990120 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:41:35.127661 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47bc3df-b326-4c4b-97d7-63b1a479823e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:41:35.129442 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47bc3df-b326-4c4b-97d7-63b1a479823e service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:41:35.153507 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 658/1322] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:41:34 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:41:38.901737 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aba62b-890f-4f6c-8e2b-2e0591ad1623 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:41:38.902258 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aba62b-890f-4f6c-8e2b-2e0591ad1623 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:41:38.902258 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aba62b-890f-4f6c-8e2b-2e0591ad1623 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:41:38.903577 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 665/1323] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:41:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:41:38.909353 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9cf28b-9208-4d21-80b5-f2eaa56201fe None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:41:38.909806 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9cf28b-9208-4d21-80b5-f2eaa56201fe None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:41:38.909898 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9cf28b-9208-4d21-80b5-f2eaa56201fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:41:38.976766 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-5a9cf28b-9208-4d21-80b5-f2eaa56201fe None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:41:38.977961 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-5a9cf28b-9208-4d21-80b5-f2eaa56201fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:41:39.038377 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:41:39.086089 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 659/1324] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:41:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:41:39.123412 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aba62b-890f-4f6c-8e2b-2e0591ad1623 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:41:39.124493 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-84aba62b-890f-4f6c-8e2b-2e0591ad1623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:41:39.238516 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aba62b-890f-4f6c-8e2b-2e0591ad1623 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:41:39.252607 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:41:39.325009 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1815a40-6d31-4a67-b7a5-6049c6172d29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:41:39.325009 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1815a40-6d31-4a67-b7a5-6049c6172d29 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:41:39.325009 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1815a40-6d31-4a67-b7a5-6049c6172d29 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:41:39.325009 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1815a40-6d31-4a67-b7a5-6049c6172d29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:41:39.325933 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-f1815a40-6d31-4a67-b7a5-6049c6172d29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:41:39.442552 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1815a40-6d31-4a67-b7a5-6049c6172d29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:41:39.444158 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1815a40-6d31-4a67-b7a5-6049c6172d29 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:41:39.464047 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 666/1325] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:41:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:41:41.729449 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e16064a-aa30-4780-9772-93c70f0c4bd9 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:41:41.729952 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e16064a-aa30-4780-9772-93c70f0c4bd9 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:41:41.729952 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e16064a-aa30-4780-9772-93c70f0c4bd9 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:41:41.731356 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 660/1326] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:41:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:41:41.736909 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c47057-befd-4cf5-8c20-9141abe8202b None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:41:41.737348 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c47057-befd-4cf5-8c20-9141abe8202b None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:41:41.737603 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c47057-befd-4cf5-8c20-9141abe8202b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:41:41.785560 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:41:41.804541 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-21c47057-befd-4cf5-8c20-9141abe8202b None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:41:41.806738 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-21c47057-befd-4cf5-8c20-9141abe8202b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:41:41.914969 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 667/1327] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:41:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:41:41.952731 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e16064a-aa30-4780-9772-93c70f0c4bd9 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:41:41.954470 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-5e16064a-aa30-4780-9772-93c70f0c4bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:41:42.078272 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e16064a-aa30-4780-9772-93c70f0c4bd9 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:41:42.161421 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0058c1-4645-41d8-9fd1-2e54d6302116 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:41:42.162066 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0058c1-4645-41d8-9fd1-2e54d6302116 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:41:42.162357 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0058c1-4645-41d8-9fd1-2e54d6302116 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:41:42.162620 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0058c1-4645-41d8-9fd1-2e54d6302116 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:41:42.164312 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-ce0058c1-4645-41d8-9fd1-2e54d6302116 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:41:42.182940 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:41:42.297822 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0058c1-4645-41d8-9fd1-2e54d6302116 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:41:42.300104 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0058c1-4645-41d8-9fd1-2e54d6302116 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:41:42.321043 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 661/1328] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:41:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:41:44.614891 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6062b913-4bc6-49c9-923c-d93ef791303b None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:41:44.615442 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6062b913-4bc6-49c9-923c-d93ef791303b None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:41:44.615442 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6062b913-4bc6-49c9-923c-d93ef791303b None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:41:44.616892 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 668/1329] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:41:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:41:44.622302 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9084d3e-dd11-42cf-a24a-4fc9e7a02bbf None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:41:44.622708 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9084d3e-dd11-42cf-a24a-4fc9e7a02bbf None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:41:44.622865 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9084d3e-dd11-42cf-a24a-4fc9e7a02bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:41:44.687831 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-a9084d3e-dd11-42cf-a24a-4fc9e7a02bbf None None] MFA Rules not processed for user `7fa722bf1210411e9101bfd161c43eaa`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:41:44.689077 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-a9084d3e-dd11-42cf-a24a-4fc9e7a02bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:41:44.757446 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:41:44.790912 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 662/1330] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:41:44 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:41:44.829075 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6062b913-4bc6-49c9-923c-d93ef791303b None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:41:44.831403 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-6062b913-4bc6-49c9-923c-d93ef791303b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:41:44.950706 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6062b913-4bc6-49c9-923c-d93ef791303b None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:41:44.973505 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:41:45.031872 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eb188f-4791-4466-9cf1-7fd630eda77a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:41:45.032619 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eb188f-4791-4466-9cf1-7fd630eda77a service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:41:45.032919 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eb188f-4791-4466-9cf1-7fd630eda77a service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:41:45.033198 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eb188f-4791-4466-9cf1-7fd630eda77a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:41:45.034993 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-48eb188f-4791-4466-9cf1-7fd630eda77a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:41:45.144359 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eb188f-4791-4466-9cf1-7fd630eda77a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:41:45.145859 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eb188f-4791-4466-9cf1-7fd630eda77a service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:41:45.163790 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 669/1331] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:41:44 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:41:47.787925 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d45df87-6e05-4d7e-b723-8e5059a9c0c0 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:41:47.788527 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d45df87-6e05-4d7e-b723-8e5059a9c0c0 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:41:47.788527 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d45df87-6e05-4d7e-b723-8e5059a9c0c0 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:41:47.789868 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 663/1332] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:41:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:41:47.794977 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf133d0-915b-46de-a7fa-6f3e69d2f977 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:41:47.795397 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf133d0-915b-46de-a7fa-6f3e69d2f977 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:41:47.795539 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf133d0-915b-46de-a7fa-6f3e69d2f977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:41:47.858192 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:41:47.861650 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-8cf133d0-915b-46de-a7fa-6f3e69d2f977 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:41:47.862915 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-8cf133d0-915b-46de-a7fa-6f3e69d2f977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:41:47.959044 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 670/1333] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:41:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:41:47.995030 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d45df87-6e05-4d7e-b723-8e5059a9c0c0 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:41:47.996590 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-9d45df87-6e05-4d7e-b723-8e5059a9c0c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:41:48.105570 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d45df87-6e05-4d7e-b723-8e5059a9c0c0 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:41:48.182059 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534a48f0-1722-417f-a81c-02e98c62cec9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:41:48.182718 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534a48f0-1722-417f-a81c-02e98c62cec9 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:41:48.183030 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534a48f0-1722-417f-a81c-02e98c62cec9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:41:48.183271 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534a48f0-1722-417f-a81c-02e98c62cec9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:41:48.185986 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-534a48f0-1722-417f-a81c-02e98c62cec9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:41:48.214893 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:41:48.296736 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534a48f0-1722-417f-a81c-02e98c62cec9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:41:48.298154 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534a48f0-1722-417f-a81c-02e98c62cec9 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:41:48.322558 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 664/1334] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:41:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:41:50.491260 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd8068-8ec5-4582-a299-6c7131f6903e None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:41:50.491822 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd8068-8ec5-4582-a299-6c7131f6903e None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:41:50.491822 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd8068-8ec5-4582-a299-6c7131f6903e None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:41:50.493290 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 671/1335] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:41:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:41:50.498571 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc6a6b5-4dfd-4232-91d9-3ee9a0a2664e None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:41:50.498968 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc6a6b5-4dfd-4232-91d9-3ee9a0a2664e None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:41:50.499120 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc6a6b5-4dfd-4232-91d9-3ee9a0a2664e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:41:50.565023 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-2bc6a6b5-4dfd-4232-91d9-3ee9a0a2664e None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:41:50.566215 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-2bc6a6b5-4dfd-4232-91d9-3ee9a0a2664e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:41:50.643171 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:41:50.665870 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 665/1336] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:41:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:41:50.706690 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dd8068-8ec5-4582-a299-6c7131f6903e None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:41:50.708272 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a6dd8068-8ec5-4582-a299-6c7131f6903e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:41:50.811192 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dd8068-8ec5-4582-a299-6c7131f6903e None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:41:50.840079 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:41:50.884886 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fae723-c10f-4022-99ca-43333b814d71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:41:50.885526 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fae723-c10f-4022-99ca-43333b814d71 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:41:50.885810 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fae723-c10f-4022-99ca-43333b814d71 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:41:50.886086 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fae723-c10f-4022-99ca-43333b814d71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:41:50.887717 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-47fae723-c10f-4022-99ca-43333b814d71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:41:50.995901 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fae723-c10f-4022-99ca-43333b814d71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:41:50.997362 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fae723-c10f-4022-99ca-43333b814d71 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:41:51.015888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 672/1337] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:41:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:41:53.711107 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c799cf-0158-4b17-b044-2bc28ea721c8 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:41:53.711667 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c799cf-0158-4b17-b044-2bc28ea721c8 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:41:53.711667 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c799cf-0158-4b17-b044-2bc28ea721c8 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:41:53.713067 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 666/1338] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:41:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:41:53.718142 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe1205-03f7-4d50-b1bf-92df299f35e5 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:41:53.718565 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe1205-03f7-4d50-b1bf-92df299f35e5 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:41:53.718696 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe1205-03f7-4d50-b1bf-92df299f35e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:41:53.783573 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-a6fe1205-03f7-4d50-b1bf-92df299f35e5 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:41:53.784808 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a6fe1205-03f7-4d50-b1bf-92df299f35e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:41:53.792924 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:41:53.884291 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 673/1339] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:41:53 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:41:53.920746 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c799cf-0158-4b17-b044-2bc28ea721c8 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:41:53.924384 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-e1c799cf-0158-4b17-b044-2bc28ea721c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:41:54.034646 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c799cf-0158-4b17-b044-2bc28ea721c8 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:41:54.047201 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8bfe73-5857-4654-846a-0c7baba90f0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:41:54.049104 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8bfe73-5857-4654-846a-0c7baba90f0b service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:41:54.049104 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8bfe73-5857-4654-846a-0c7baba90f0b service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:41:54.049104 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8bfe73-5857-4654-846a-0c7baba90f0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:41:54.050627 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-6b8bfe73-5857-4654-846a-0c7baba90f0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:41:54.150913 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:41:54.153123 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8bfe73-5857-4654-846a-0c7baba90f0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:41:54.154551 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8bfe73-5857-4654-846a-0c7baba90f0b service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:41:54.174699 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 667/1340] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:41:53 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:41:58.699038 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f86cd6b-d896-419d-a5f8-8636798aa0bb None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:41:58.699580 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f86cd6b-d896-419d-a5f8-8636798aa0bb None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:41:58.699685 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f86cd6b-d896-419d-a5f8-8636798aa0bb None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:41:58.701149 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 674/1341] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:41:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:41:58.707092 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa5605-27fa-4afc-bed9-b8fddca44da7 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:41:58.707567 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa5605-27fa-4afc-bed9-b8fddca44da7 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:41:58.707784 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa5605-27fa-4afc-bed9-b8fddca44da7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:41:58.755526 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-7afa5605-27fa-4afc-bed9-b8fddca44da7 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:41:58.756770 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-7afa5605-27fa-4afc-bed9-b8fddca44da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:41:58.835941 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 668/1342] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:41:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:41:58.875317 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f86cd6b-d896-419d-a5f8-8636798aa0bb None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:41:58.878524 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-5f86cd6b-d896-419d-a5f8-8636798aa0bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:41:58.902812 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f86cd6b-d896-419d-a5f8-8636798aa0bb None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:41:58.909168 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e96870c-c24a-4864-b06c-4aafeab5101e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:41:58.909168 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e96870c-c24a-4864-b06c-4aafeab5101e service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:41:58.909168 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e96870c-c24a-4864-b06c-4aafeab5101e service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:41:58.909168 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e96870c-c24a-4864-b06c-4aafeab5101e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:41:58.918226 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e96870c-c24a-4864-b06c-4aafeab5101e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:41:58.919645 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e96870c-c24a-4864-b06c-4aafeab5101e service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:41:58.930484 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 675/1343] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:41:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:42:02.080986 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc26437d-1cb6-4e73-a0e8-9a643bc332c1 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:02.081571 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc26437d-1cb6-4e73-a0e8-9a643bc332c1 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:02.081571 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc26437d-1cb6-4e73-a0e8-9a643bc332c1 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:02.083044 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 669/1344] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:42:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:42:02.088994 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1de8e05-a57c-4423-b8ee-9efe36efbd8e None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:02.093688 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1de8e05-a57c-4423-b8ee-9efe36efbd8e None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:02.093688 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1de8e05-a57c-4423-b8ee-9efe36efbd8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:02.120789 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-c1de8e05-a57c-4423-b8ee-9efe36efbd8e None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:02.121988 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c1de8e05-a57c-4423-b8ee-9efe36efbd8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:02.130035 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 676/1345] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:42:02 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:02.167804 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc26437d-1cb6-4e73-a0e8-9a643bc332c1 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:02.179034 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc26437d-1cb6-4e73-a0e8-9a643bc332c1 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:02.182491 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81374381-071e-4cc3-a433-80c29b4cd3bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:02.183125 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81374381-071e-4cc3-a433-80c29b4cd3bb service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:02.183407 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81374381-071e-4cc3-a433-80c29b4cd3bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:02.183684 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81374381-071e-4cc3-a433-80c29b4cd3bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:02.192844 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81374381-071e-4cc3-a433-80c29b4cd3bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:02.194271 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81374381-071e-4cc3-a433-80c29b4cd3bb service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:02.203551 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 670/1346] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:42:02 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:42:06.662917 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9714c9-fab8-47d3-beaa-902433469116 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:06.663479 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9714c9-fab8-47d3-beaa-902433469116 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:06.663479 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9714c9-fab8-47d3-beaa-902433469116 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:06.664895 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 677/1347] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:42:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:42:06.670134 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d852f-301c-432c-8477-79cafe0ced9f None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:06.670575 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d852f-301c-432c-8477-79cafe0ced9f None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:06.670682 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d852f-301c-432c-8477-79cafe0ced9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:06.689012 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-386d852f-301c-432c-8477-79cafe0ced9f None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:06.690229 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-386d852f-301c-432c-8477-79cafe0ced9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:06.698332 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 671/1348] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:42:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:06.736959 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9714c9-fab8-47d3-beaa-902433469116 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:06.747592 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9714c9-fab8-47d3-beaa-902433469116 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:06.751685 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5d931a-e889-4efc-a012-ac3584b5725a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:06.752361 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5d931a-e889-4efc-a012-ac3584b5725a service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:06.752645 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5d931a-e889-4efc-a012-ac3584b5725a service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:06.752916 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5d931a-e889-4efc-a012-ac3584b5725a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:06.761902 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5d931a-e889-4efc-a012-ac3584b5725a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:06.763328 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5d931a-e889-4efc-a012-ac3584b5725a service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:06.775723 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 678/1349] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:42:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:42:11.847924 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7cbc87-6d6c-4920-bec0-896b4e9ebd4b None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:11.848453 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7cbc87-6d6c-4920-bec0-896b4e9ebd4b None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:11.848545 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7cbc87-6d6c-4920-bec0-896b4e9ebd4b None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:11.850149 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 672/1350] 10.222.0.42 () {54 vars in 921 bytes} [Wed Jun 15 14:42:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:42:11.855196 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73ad419-462a-4d8b-8865-8b81f74086bf None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:11.855944 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73ad419-462a-4d8b-8865-8b81f74086bf None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:11.856413 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73ad419-462a-4d8b-8865-8b81f74086bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:11.930862 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-f73ad419-462a-4d8b-8865-8b81f74086bf None None] MFA Rules not processed for user `2bb6671c19464424a6e9cd4519ac93e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:11.935081 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-f73ad419-462a-4d8b-8865-8b81f74086bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:11.979472 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:42:12.021959 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 679/1351] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Jun 15 14:42:11 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:16.135352 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea6661b-a463-45f3-8e16-e95937d8af21 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:16.136006 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea6661b-a463-45f3-8e16-e95937d8af21 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:16.136006 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea6661b-a463-45f3-8e16-e95937d8af21 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:16.137309 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 673/1352] 10.222.0.42 () {54 vars in 921 bytes} [Wed Jun 15 14:42:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:42:16.143713 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab219256-9489-4e4c-8b6c-7a84b9ec2fc9 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:16.144167 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab219256-9489-4e4c-8b6c-7a84b9ec2fc9 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:16.144307 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab219256-9489-4e4c-8b6c-7a84b9ec2fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:16.163509 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-ab219256-9489-4e4c-8b6c-7a84b9ec2fc9 None None] MFA Rules not processed for user `2bb6671c19464424a6e9cd4519ac93e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:16.164759 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-ab219256-9489-4e4c-8b6c-7a84b9ec2fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:16.179333 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 680/1353] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Jun 15 14:42:16 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:16.685992 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=60 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:42:17.845756 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f79212-7aa0-4d20-ba03-1fe93bf35af0 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:17.846418 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f79212-7aa0-4d20-ba03-1fe93bf35af0 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:17.846418 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f79212-7aa0-4d20-ba03-1fe93bf35af0 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:17.847735 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 674/1354] 10.222.0.42 () {54 vars in 921 bytes} [Wed Jun 15 14:42:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:42:17.852556 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e77cb03-ce29-4aa6-a960-c6d4571ef520 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:17.852987 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e77cb03-ce29-4aa6-a960-c6d4571ef520 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:17.855425 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e77cb03-ce29-4aa6-a960-c6d4571ef520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:17.871234 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-4e77cb03-ce29-4aa6-a960-c6d4571ef520 None None] MFA Rules not processed for user `2bb6671c19464424a6e9cd4519ac93e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:17.872531 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-4e77cb03-ce29-4aa6-a960-c6d4571ef520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:17.882288 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 681/1355] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Jun 15 14:42:17 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:42:21.830122 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6b990b-12fa-4c4c-a38e-fabdc06951d2 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:21.830653 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6b990b-12fa-4c4c-a38e-fabdc06951d2 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:21.830653 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6b990b-12fa-4c4c-a38e-fabdc06951d2 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:21.831916 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 675/1356] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:42:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:42:21.837525 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d30fe5f-7852-4e83-a58e-6d078fe7f965 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:21.837974 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d30fe5f-7852-4e83-a58e-6d078fe7f965 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:21.838066 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d30fe5f-7852-4e83-a58e-6d078fe7f965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:21.856611 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-4d30fe5f-7852-4e83-a58e-6d078fe7f965 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:21.858072 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-4d30fe5f-7852-4e83-a58e-6d078fe7f965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:21.866634 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 682/1357] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:42:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:21.923517 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2db8c0-40ea-4b13-8e72-2b94324da565 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:21.923898 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2db8c0-40ea-4b13-8e72-2b94324da565 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:21.924185 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2db8c0-40ea-4b13-8e72-2b94324da565 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:21.925753 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 676/1358] 10.222.0.42 () {54 vars in 961 bytes} [Wed Jun 15 14:42:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:42:21.933224 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ade4d0-e480-44c5-a4fc-abd9b47680c4 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:21.933822 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ade4d0-e480-44c5-a4fc-abd9b47680c4 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:21.934269 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ade4d0-e480-44c5-a4fc-abd9b47680c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:21.988597 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-40ade4d0-e480-44c5-a4fc-abd9b47680c4 None None] MFA Rules not processed for user `0927c05c340c48e48b18754888aa9c35`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:21.989781 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-40ade4d0-e480-44c5-a4fc-abd9b47680c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:22.087049 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 683/1359] 10.222.0.42 () {58 vars in 1058 bytes} [Wed Jun 15 14:42:21 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:22.095781 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2db8c0-40ea-4b13-8e72-2b94324da565 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:22.112806 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2db8c0-40ea-4b13-8e72-2b94324da565 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:22.118515 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49ba289-3fe6-42cf-8b65-ca29c63dcc40 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0927c05c340c48e48b18754888aa9c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:22.122383 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ba289-3fe6-42cf-8b65-ca29c63dcc40 service nova] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:22.122383 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ba289-3fe6-42cf-8b65-ca29c63dcc40 service nova] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:22.122383 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ba289-3fe6-42cf-8b65-ca29c63dcc40 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:22.129408 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49ba289-3fe6-42cf-8b65-ca29c63dcc40 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:22.131127 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49ba289-3fe6-42cf-8b65-ca29c63dcc40 service nova] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:22.143922 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 677/1360] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:42:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:42:24.513810 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2123bbb3-354e-41d0-b486-3326d04194e2 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:24.514613 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2123bbb3-354e-41d0-b486-3326d04194e2 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:24.514613 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2123bbb3-354e-41d0-b486-3326d04194e2 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:24.516929 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 684/1361] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:42:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:42:24.522226 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a47bdc-91fe-4f4e-8598-b1ecf1c7c3a9 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:24.522692 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a47bdc-91fe-4f4e-8598-b1ecf1c7c3a9 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:24.522809 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a47bdc-91fe-4f4e-8598-b1ecf1c7c3a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:24.541775 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-48a47bdc-91fe-4f4e-8598-b1ecf1c7c3a9 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:24.542970 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-48a47bdc-91fe-4f4e-8598-b1ecf1c7c3a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:24.557068 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 678/1362] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:42:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:24.600972 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2123bbb3-354e-41d0-b486-3326d04194e2 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:24.617578 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2123bbb3-354e-41d0-b486-3326d04194e2 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:24.623322 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdbf094-47e2-47c5-8075-de5e1be7063f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0927c05c340c48e48b18754888aa9c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:24.624248 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdbf094-47e2-47c5-8075-de5e1be7063f service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:24.624550 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdbf094-47e2-47c5-8075-de5e1be7063f service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:24.624816 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdbf094-47e2-47c5-8075-de5e1be7063f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:24.635386 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdbf094-47e2-47c5-8075-de5e1be7063f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:24.637176 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdbf094-47e2-47c5-8075-de5e1be7063f service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:24.649794 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 685/1363] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:42:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:42:27.280010 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d690d22a-1f3a-4cac-8012-25eafad6c38a None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:27.280567 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d690d22a-1f3a-4cac-8012-25eafad6c38a None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:27.280676 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d690d22a-1f3a-4cac-8012-25eafad6c38a None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:27.282741 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 679/1364] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:42:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:42:27.288300 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7c8e4-e774-4a31-84e0-3c61a90c7040 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:27.288816 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7c8e4-e774-4a31-84e0-3c61a90c7040 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:27.288922 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7c8e4-e774-4a31-84e0-3c61a90c7040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:27.309708 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-a4a7c8e4-e774-4a31-84e0-3c61a90c7040 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:27.311304 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a4a7c8e4-e774-4a31-84e0-3c61a90c7040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:27.321384 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 686/1365] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:42:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:27.373405 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28cd49-3081-4b8a-ac5c-be562a864465 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:27.374160 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28cd49-3081-4b8a-ac5c-be562a864465 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:27.374452 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28cd49-3081-4b8a-ac5c-be562a864465 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:27.377001 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 680/1366] 10.222.0.42 () {54 vars in 961 bytes} [Wed Jun 15 14:42:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:42:27.384196 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668718e0-7acc-42b3-8033-e6f8f9c9d84f None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:27.384196 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668718e0-7acc-42b3-8033-e6f8f9c9d84f None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:27.384196 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668718e0-7acc-42b3-8033-e6f8f9c9d84f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:27.403570 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-668718e0-7acc-42b3-8033-e6f8f9c9d84f None None] MFA Rules not processed for user `0927c05c340c48e48b18754888aa9c35`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:27.404859 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-668718e0-7acc-42b3-8033-e6f8f9c9d84f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:27.417684 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 687/1367] 10.222.0.42 () {58 vars in 1058 bytes} [Wed Jun 15 14:42:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:42:27.426077 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c28cd49-3081-4b8a-ac5c-be562a864465 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:27.442561 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c28cd49-3081-4b8a-ac5c-be562a864465 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:27.448115 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336d01d7-c2aa-4128-843e-3f82b6f8fb1b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0927c05c340c48e48b18754888aa9c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:27.448775 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d01d7-c2aa-4128-843e-3f82b6f8fb1b service nova] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:27.449056 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d01d7-c2aa-4128-843e-3f82b6f8fb1b service nova] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:27.449331 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d01d7-c2aa-4128-843e-3f82b6f8fb1b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:27.458462 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336d01d7-c2aa-4128-843e-3f82b6f8fb1b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:27.460053 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336d01d7-c2aa-4128-843e-3f82b6f8fb1b service nova] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:27.471591 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 681/1368] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:42:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:42:30.015165 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e9240c-4128-4766-8a3d-59447e4240be None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:30.015740 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e9240c-4128-4766-8a3d-59447e4240be None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:30.015740 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e9240c-4128-4766-8a3d-59447e4240be None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:30.017747 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 688/1369] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:42:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:42:30.023194 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c48facf-16e0-4950-bd3f-255652b23d74 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:30.023646 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c48facf-16e0-4950-bd3f-255652b23d74 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:30.023756 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c48facf-16e0-4950-bd3f-255652b23d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:30.043068 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-6c48facf-16e0-4950-bd3f-255652b23d74 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:30.044292 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-6c48facf-16e0-4950-bd3f-255652b23d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:30.053400 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 682/1370] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:42:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:30.098744 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e9240c-4128-4766-8a3d-59447e4240be None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:30.114780 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e9240c-4128-4766-8a3d-59447e4240be None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:30.120659 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b99b7cd-63bd-482f-ab67-165b6ed171c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0927c05c340c48e48b18754888aa9c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:30.121496 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b99b7cd-63bd-482f-ab67-165b6ed171c8 service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:30.121783 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b99b7cd-63bd-482f-ab67-165b6ed171c8 service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:30.122057 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b99b7cd-63bd-482f-ab67-165b6ed171c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:30.132853 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b99b7cd-63bd-482f-ab67-165b6ed171c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:30.134484 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b99b7cd-63bd-482f-ab67-165b6ed171c8 service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:30.146907 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 689/1371] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:42:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:42:30.192544 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b5712f-39f9-4fa4-a7af-ed20cd652d32 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:30.192821 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b5712f-39f9-4fa4-a7af-ed20cd652d32 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:30.193127 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b5712f-39f9-4fa4-a7af-ed20cd652d32 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:30.194658 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 683/1372] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:42:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:42:30.200000 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea99977-b83d-44c4-abfd-1d7540f0e3cd None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:30.200411 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea99977-b83d-44c4-abfd-1d7540f0e3cd None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:30.200755 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea99977-b83d-44c4-abfd-1d7540f0e3cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:30.219766 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-5ea99977-b83d-44c4-abfd-1d7540f0e3cd None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:30.221313 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-5ea99977-b83d-44c4-abfd-1d7540f0e3cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:30.229947 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 690/1373] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:42:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:42:30.545354 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78f3759-7d15-4f84-b0dd-9c0fa4c45516 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:30.545354 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78f3759-7d15-4f84-b0dd-9c0fa4c45516 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:30.545354 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78f3759-7d15-4f84-b0dd-9c0fa4c45516 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:30.555518 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 684/1374] 10.222.0.42 () {54 vars in 963 bytes} [Wed Jun 15 14:42:30 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:42:30.561149 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41164bf1-33b2-4f2a-b7db-0dbc7b822afd None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:30.561517 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41164bf1-33b2-4f2a-b7db-0dbc7b822afd None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:30.561866 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41164bf1-33b2-4f2a-b7db-0dbc7b822afd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:30.590010 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:42:30.616740 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-41164bf1-33b2-4f2a-b7db-0dbc7b822afd None None] MFA Rules not processed for user `e44f6cd435be4e03bc79cb3125d71451`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:30.617945 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-41164bf1-33b2-4f2a-b7db-0dbc7b822afd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:30.686132 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 691/1375] 10.222.0.42 () {58 vars in 1060 bytes} [Wed Jun 15 14:42:30 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:42:30.694560 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78f3759-7d15-4f84-b0dd-9c0fa4c45516 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:30.707741 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78f3759-7d15-4f84-b0dd-9c0fa4c45516 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:30.711174 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50be145b-1b08-4a4b-be59-9608646fa88d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:30.711811 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be145b-1b08-4a4b-be59-9608646fa88d service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:30.712111 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be145b-1b08-4a4b-be59-9608646fa88d service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:30.712394 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be145b-1b08-4a4b-be59-9608646fa88d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:30.722305 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50be145b-1b08-4a4b-be59-9608646fa88d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:30.724123 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50be145b-1b08-4a4b-be59-9608646fa88d service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:30.734143 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 685/1376] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:42:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:42:32.452586 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c0db78-3082-4d8c-a11a-2b9582fd55ad None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:32.453212 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c0db78-3082-4d8c-a11a-2b9582fd55ad None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:32.453212 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c0db78-3082-4d8c-a11a-2b9582fd55ad None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:32.455192 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 692/1377] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:42:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:42:32.461584 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125e64c4-5dcf-482a-80be-67cd9c45c1f9 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:32.462151 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125e64c4-5dcf-482a-80be-67cd9c45c1f9 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:32.462151 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125e64c4-5dcf-482a-80be-67cd9c45c1f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:32.481975 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-125e64c4-5dcf-482a-80be-67cd9c45c1f9 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:32.483217 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-125e64c4-5dcf-482a-80be-67cd9c45c1f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:32.491843 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 686/1378] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:42:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:42:32.538149 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c0db78-3082-4d8c-a11a-2b9582fd55ad None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:32.554051 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c0db78-3082-4d8c-a11a-2b9582fd55ad None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:32.559752 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a825db2-dd66-47c9-9dcc-48cbd18ddf17 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0927c05c340c48e48b18754888aa9c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:32.560468 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a825db2-dd66-47c9-9dcc-48cbd18ddf17 service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:32.560751 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a825db2-dd66-47c9-9dcc-48cbd18ddf17 service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:32.561030 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a825db2-dd66-47c9-9dcc-48cbd18ddf17 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:32.570735 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a825db2-dd66-47c9-9dcc-48cbd18ddf17 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:32.572469 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a825db2-dd66-47c9-9dcc-48cbd18ddf17 service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:32.583730 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 693/1379] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:42:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:42:34.903718 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e318e430-3f86-4b60-9997-3b3d26a7878e None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:34.904303 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e318e430-3f86-4b60-9997-3b3d26a7878e None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:34.904634 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e318e430-3f86-4b60-9997-3b3d26a7878e None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:34.906121 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 687/1380] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:42:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:42:34.912509 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d1d6ca-3f8b-40e0-88fd-1c15a0e56b7b None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:34.913018 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d1d6ca-3f8b-40e0-88fd-1c15a0e56b7b None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:34.913102 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d1d6ca-3f8b-40e0-88fd-1c15a0e56b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:34.932252 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-20d1d6ca-3f8b-40e0-88fd-1c15a0e56b7b None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:34.933477 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-20d1d6ca-3f8b-40e0-88fd-1c15a0e56b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:34.942514 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 694/1381] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:42:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:34.988504 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e318e430-3f86-4b60-9997-3b3d26a7878e None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:35.004272 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e318e430-3f86-4b60-9997-3b3d26a7878e None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:35.010162 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93062c38-166f-4b44-9838-6693e22021b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0927c05c340c48e48b18754888aa9c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:35.011076 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93062c38-166f-4b44-9838-6693e22021b5 service nova] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:35.011459 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93062c38-166f-4b44-9838-6693e22021b5 service nova] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:35.011839 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93062c38-166f-4b44-9838-6693e22021b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:35.021180 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93062c38-166f-4b44-9838-6693e22021b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:35.022886 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93062c38-166f-4b44-9838-6693e22021b5 service nova] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:35.034253 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 688/1382] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:42:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:42:37.383016 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb71fd5-8c33-41cb-befa-afec35a7ec37 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:37.383553 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb71fd5-8c33-41cb-befa-afec35a7ec37 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:37.383553 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb71fd5-8c33-41cb-befa-afec35a7ec37 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:37.389547 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 695/1383] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:42:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:42:37.391979 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa9ce7-beae-4447-8de6-80f452467304 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:37.393578 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa9ce7-beae-4447-8de6-80f452467304 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:37.394096 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa9ce7-beae-4447-8de6-80f452467304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:37.415605 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-20fa9ce7-beae-4447-8de6-80f452467304 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:37.417236 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-20fa9ce7-beae-4447-8de6-80f452467304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:37.427365 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 689/1384] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:42:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:37.469385 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb71fd5-8c33-41cb-befa-afec35a7ec37 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:37.488451 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb71fd5-8c33-41cb-befa-afec35a7ec37 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:37.495054 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd204ac-6fff-4794-a1ac-71577b3c0b08 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0927c05c340c48e48b18754888aa9c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:37.496061 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd204ac-6fff-4794-a1ac-71577b3c0b08 service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:37.496599 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd204ac-6fff-4794-a1ac-71577b3c0b08 service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:37.497076 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd204ac-6fff-4794-a1ac-71577b3c0b08 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:37.508147 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd204ac-6fff-4794-a1ac-71577b3c0b08 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:37.510122 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd204ac-6fff-4794-a1ac-71577b3c0b08 service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:37.527397 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 696/1385] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:42:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:42:39.960530 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d338ea5-82f3-4df8-b8b4-ad1f468a4e87 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:39.961064 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d338ea5-82f3-4df8-b8b4-ad1f468a4e87 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:39.961064 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d338ea5-82f3-4df8-b8b4-ad1f468a4e87 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:39.962374 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 690/1386] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:42:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:42:39.967729 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdc70f4-4103-4b91-9416-03eb6fdfe3b0 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:39.968161 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdc70f4-4103-4b91-9416-03eb6fdfe3b0 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:39.968352 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdc70f4-4103-4b91-9416-03eb6fdfe3b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:39.986951 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-7bdc70f4-4103-4b91-9416-03eb6fdfe3b0 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:39.988201 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-7bdc70f4-4103-4b91-9416-03eb6fdfe3b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:39.996177 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 697/1387] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:42:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:40.044545 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d338ea5-82f3-4df8-b8b4-ad1f468a4e87 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:40.060111 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d338ea5-82f3-4df8-b8b4-ad1f468a4e87 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:40.066713 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62f20b8-ada6-4b65-afda-a412ac5b27a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0927c05c340c48e48b18754888aa9c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:40.067346 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f20b8-ada6-4b65-afda-a412ac5b27a0 service nova] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:40.067626 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f20b8-ada6-4b65-afda-a412ac5b27a0 service nova] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:40.067897 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f20b8-ada6-4b65-afda-a412ac5b27a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:40.077991 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62f20b8-ada6-4b65-afda-a412ac5b27a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:40.079901 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62f20b8-ada6-4b65-afda-a412ac5b27a0 service nova] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:40.089776 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 691/1388] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:42:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:42:42.423776 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14afe299-b643-440d-9a65-a03a825e2bf8 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:42.424315 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14afe299-b643-440d-9a65-a03a825e2bf8 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:42.424415 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14afe299-b643-440d-9a65-a03a825e2bf8 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:42.425742 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 698/1389] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:42:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:42:42.431947 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc0b2c-3943-4765-b0d9-d1f13403518b None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:42.432560 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc0b2c-3943-4765-b0d9-d1f13403518b None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:42.432560 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc0b2c-3943-4765-b0d9-d1f13403518b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:42.450178 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-e5bc0b2c-3943-4765-b0d9-d1f13403518b None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:42.451372 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-e5bc0b2c-3943-4765-b0d9-d1f13403518b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:42.460447 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 692/1390] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:42:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:42.507465 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14afe299-b643-440d-9a65-a03a825e2bf8 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:42.527086 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14afe299-b643-440d-9a65-a03a825e2bf8 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:42.534547 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8a374d-fc52-4dd7-9988-b00c0eec4f71 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0927c05c340c48e48b18754888aa9c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:42.535419 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a374d-fc52-4dd7-9988-b00c0eec4f71 service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:42.535789 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a374d-fc52-4dd7-9988-b00c0eec4f71 service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:42.536173 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a374d-fc52-4dd7-9988-b00c0eec4f71 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:42.550409 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8a374d-fc52-4dd7-9988-b00c0eec4f71 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:42.552749 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8a374d-fc52-4dd7-9988-b00c0eec4f71 service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:42.566437 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 699/1391] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:42:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:42:45.246795 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62cbcfd-4459-4101-9922-8c8579ebfb73 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:45.247539 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62cbcfd-4459-4101-9922-8c8579ebfb73 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:45.247539 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62cbcfd-4459-4101-9922-8c8579ebfb73 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:45.248860 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 693/1392] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:42:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:42:45.254766 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c7d424-6e76-4c1d-a93f-4129ae0fe38c None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:45.255231 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c7d424-6e76-4c1d-a93f-4129ae0fe38c None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:45.255322 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c7d424-6e76-4c1d-a93f-4129ae0fe38c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:45.274668 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-f5c7d424-6e76-4c1d-a93f-4129ae0fe38c None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:45.275906 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-f5c7d424-6e76-4c1d-a93f-4129ae0fe38c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:45.284850 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 700/1393] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:42:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:45.329878 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62cbcfd-4459-4101-9922-8c8579ebfb73 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:45.348734 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62cbcfd-4459-4101-9922-8c8579ebfb73 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:45.354731 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7244d2a3-4cb6-4a03-bde9-77b7e09ca045 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0927c05c340c48e48b18754888aa9c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:45.355393 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7244d2a3-4cb6-4a03-bde9-77b7e09ca045 service nova] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:45.355682 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7244d2a3-4cb6-4a03-bde9-77b7e09ca045 service nova] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:45.355961 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7244d2a3-4cb6-4a03-bde9-77b7e09ca045 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:45.365516 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7244d2a3-4cb6-4a03-bde9-77b7e09ca045 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:45.367128 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7244d2a3-4cb6-4a03-bde9-77b7e09ca045 service nova] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:45.378431 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 694/1394] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:42:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:42:46.573722 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bdee70-b7af-49ea-a800-3d4be3979270 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:46.574256 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bdee70-b7af-49ea-a800-3d4be3979270 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:46.574256 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bdee70-b7af-49ea-a800-3d4be3979270 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:46.575857 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 701/1395] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:42:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:42:46.581384 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af99065e-6805-4842-942b-cda8a4932fdd None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:46.581819 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af99065e-6805-4842-942b-cda8a4932fdd None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:46.581930 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af99065e-6805-4842-942b-cda8a4932fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:46.599920 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-af99065e-6805-4842-942b-cda8a4932fdd None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:46.601154 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-af99065e-6805-4842-942b-cda8a4932fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:46.608949 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 695/1396] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:42:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:46.666915 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04993aa-6452-4a4f-8fea-67ea9471632b None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:46.667210 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04993aa-6452-4a4f-8fea-67ea9471632b None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:46.667490 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04993aa-6452-4a4f-8fea-67ea9471632b None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:46.668902 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 702/1397] 10.222.0.42 () {54 vars in 962 bytes} [Wed Jun 15 14:42:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:42:46.674444 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346c249-e659-4e14-97dd-1b65f3314193 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:46.674730 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346c249-e659-4e14-97dd-1b65f3314193 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:46.675012 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346c249-e659-4e14-97dd-1b65f3314193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:46.728022 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-8346c249-e659-4e14-97dd-1b65f3314193 None None] MFA Rules not processed for user `4f18c57f27784694935bfb0bcea48a78`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:46.729278 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-8346c249-e659-4e14-97dd-1b65f3314193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:46.733619 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=69 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:42:46.792802 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 696/1398] 10.222.0.42 () {58 vars in 1059 bytes} [Wed Jun 15 14:42:46 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:46.800874 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04993aa-6452-4a4f-8fea-67ea9471632b None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:46.811655 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04993aa-6452-4a4f-8fea-67ea9471632b None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:46.816018 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5043f3-79f3-48d7-893e-81895e514caa service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f18c57f27784694935bfb0bcea48a78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:46.816881 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5043f3-79f3-48d7-893e-81895e514caa service glance] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:46.817166 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5043f3-79f3-48d7-893e-81895e514caa service glance] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:46.817442 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5043f3-79f3-48d7-893e-81895e514caa service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:46.828655 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5043f3-79f3-48d7-893e-81895e514caa service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:46.830410 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5043f3-79f3-48d7-893e-81895e514caa service glance] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:46.841122 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 703/1399] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:42:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:42:47.730669 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104cf6a1-733b-4327-84e0-1e4f4aadb85f None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:47.734150 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104cf6a1-733b-4327-84e0-1e4f4aadb85f None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:47.734150 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104cf6a1-733b-4327-84e0-1e4f4aadb85f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:47.802981 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-104cf6a1-733b-4327-84e0-1e4f4aadb85f None None] MFA Rules not processed for user `19d82c04f9994654adfb10f4bd0573c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:47.804459 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-104cf6a1-733b-4327-84e0-1e4f4aadb85f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:47.808182 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85824061-412a-4a8d-a693-6a5fd5c2486c None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:47.808692 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85824061-412a-4a8d-a693-6a5fd5c2486c None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:47.808786 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85824061-412a-4a8d-a693-6a5fd5c2486c None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:47.810434 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 704/1400] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:42:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:42:47.816850 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5d97d8-c9c0-4604-afcd-780ab74fb5f2 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:47.817138 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5d97d8-c9c0-4604-afcd-780ab74fb5f2 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:47.817438 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5d97d8-c9c0-4604-afcd-780ab74fb5f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:47.837648 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-7d5d97d8-c9c0-4604-afcd-780ab74fb5f2 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:47.838907 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-7d5d97d8-c9c0-4604-afcd-780ab74fb5f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:47.848623 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 705/1401] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:42:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:42:47.877532 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 697/1402] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Jun 15 14:42:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:47.895857 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5d97d8-c9c0-4604-afcd-780ab74fb5f2 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:47.912488 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5d97d8-c9c0-4604-afcd-780ab74fb5f2 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:47.920912 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae35dbc-0a11-46fb-8c1f-8fc118161003 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0927c05c340c48e48b18754888aa9c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:47.921457 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae35dbc-0a11-46fb-8c1f-8fc118161003 service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:47.921823 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae35dbc-0a11-46fb-8c1f-8fc118161003 service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:47.922158 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae35dbc-0a11-46fb-8c1f-8fc118161003 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:47.934625 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3204969-e7b0-45ff-bcd6-88300217fba7 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:47.936194 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae35dbc-0a11-46fb-8c1f-8fc118161003 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:47.936318 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3204969-e7b0-45ff-bcd6-88300217fba7 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:47.936318 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3204969-e7b0-45ff-bcd6-88300217fba7 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:47.936549 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae35dbc-0a11-46fb-8c1f-8fc118161003 service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:47.936753 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 698/1403] 10.222.0.42 () {54 vars in 942 bytes} [Wed Jun 15 14:42:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:42:47.971170 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e06ed90-820c-4881-9d3a-8cd5e89ddc14 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:47.971170 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e06ed90-820c-4881-9d3a-8cd5e89ddc14 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:47.971170 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e06ed90-820c-4881-9d3a-8cd5e89ddc14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:47.971670 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 706/1404] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:42:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:42:48.064702 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-3e06ed90-820c-4881-9d3a-8cd5e89ddc14 None None] MFA Rules not processed for user `c04dc5832333498e96a7e02bbfb9484c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:48.064702 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-3e06ed90-820c-4881-9d3a-8cd5e89ddc14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:48.072982 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:42:48.102370 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 699/1405] 10.222.0.42 () {58 vars in 1039 bytes} [Wed Jun 15 14:42:47 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:48.113260 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae35dbc-0a11-46fb-8c1f-8fc118161003 service nova] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:48.127793 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae35dbc-0a11-46fb-8c1f-8fc118161003 service nova] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:48.133892 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956b09f2-98e2-462c-ae9f-0b6256d9f249 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04dc5832333498e96a7e02bbfb9484c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:48.134600 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b09f2-98e2-462c-ae9f-0b6256d9f249 service swift] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:48.134917 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b09f2-98e2-462c-ae9f-0b6256d9f249 service swift] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:48.135223 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b09f2-98e2-462c-ae9f-0b6256d9f249 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:48.148320 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956b09f2-98e2-462c-ae9f-0b6256d9f249 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:48.150189 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956b09f2-98e2-462c-ae9f-0b6256d9f249 service swift] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:48.168467 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 707/1406] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Jun 15 14:42:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 14:42:50.609845 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aabf80-fcc3-4b38-a055-7a928baff6b5 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:50.610410 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aabf80-fcc3-4b38-a055-7a928baff6b5 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:50.610410 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aabf80-fcc3-4b38-a055-7a928baff6b5 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:50.616288 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 700/1407] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:42:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:42:50.618199 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340dd70c-247d-4d01-b246-1528aec9e3cf None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:50.618777 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340dd70c-247d-4d01-b246-1528aec9e3cf None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:50.619084 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340dd70c-247d-4d01-b246-1528aec9e3cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:50.641340 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-340dd70c-247d-4d01-b246-1528aec9e3cf None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:50.643008 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-340dd70c-247d-4d01-b246-1528aec9e3cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:50.664151 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 708/1408] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:42:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:50.732292 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aabf80-fcc3-4b38-a055-7a928baff6b5 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:50.756173 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aabf80-fcc3-4b38-a055-7a928baff6b5 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:50.764959 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a263a7-8661-4fef-be78-3e8e8231bcde service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0927c05c340c48e48b18754888aa9c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:50.766956 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a263a7-8661-4fef-be78-3e8e8231bcde service nova] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:50.767435 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a263a7-8661-4fef-be78-3e8e8231bcde service nova] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:50.767847 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a263a7-8661-4fef-be78-3e8e8231bcde service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:50.782120 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a263a7-8661-4fef-be78-3e8e8231bcde service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:50.784686 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a263a7-8661-4fef-be78-3e8e8231bcde service nova] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:50.801046 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 701/1409] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:42:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:00.634159 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=82 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:00.793798 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=13 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:13.255905 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0378ca-4858-47ee-9221-940c728c2d7a None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:13.256574 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0378ca-4858-47ee-9221-940c728c2d7a None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:13.256574 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0378ca-4858-47ee-9221-940c728c2d7a None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:13.258044 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 709/1410] 10.222.0.42 () {54 vars in 918 bytes} [Wed Jun 15 14:43:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:43:13.263866 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f61a5f-0c1e-4679-b031-784a3cbbbaf8 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:13.264484 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f61a5f-0c1e-4679-b031-784a3cbbbaf8 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:13.264484 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f61a5f-0c1e-4679-b031-784a3cbbbaf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:13.285991 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-c7f61a5f-0c1e-4679-b031-784a3cbbbaf8 None None] MFA Rules not processed for user `2bb6671c19464424a6e9cd4519ac93e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:13.287233 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-c7f61a5f-0c1e-4679-b031-784a3cbbbaf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:13.297548 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 702/1411] 10.222.0.42 () {58 vars in 1015 bytes} [Wed Jun 15 14:43:13 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:43:13.316378 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1229900d-78ee-4eb3-982a-500855c80e4c None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:13.316722 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1229900d-78ee-4eb3-982a-500855c80e4c None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:13.317062 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1229900d-78ee-4eb3-982a-500855c80e4c None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:13.318665 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 710/1412] 10.222.0.42 () {54 vars in 918 bytes} [Wed Jun 15 14:43:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:43:13.323928 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946e1a57-5879-4b7c-b1c0-80c42ad2599e None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:13.324247 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946e1a57-5879-4b7c-b1c0-80c42ad2599e None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:13.324527 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946e1a57-5879-4b7c-b1c0-80c42ad2599e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:13.345405 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-946e1a57-5879-4b7c-b1c0-80c42ad2599e None None] MFA Rules not processed for user `0927c05c340c48e48b18754888aa9c35`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:13.346701 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-946e1a57-5879-4b7c-b1c0-80c42ad2599e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:13.360493 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 703/1413] 10.222.0.42 () {58 vars in 1015 bytes} [Wed Jun 15 14:43:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:15.994282 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6cbc6-fc5c-4419-850a-054effda05f2 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:15.994799 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6cbc6-fc5c-4419-850a-054effda05f2 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:15.994913 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6cbc6-fc5c-4419-850a-054effda05f2 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:15.996430 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 711/1414] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 14:43:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:43:16.001989 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b789493-3d64-47c5-bd48-9bb7732617c1 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:16.002700 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b789493-3d64-47c5-bd48-9bb7732617c1 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:16.002700 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b789493-3d64-47c5-bd48-9bb7732617c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:16.021443 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-9b789493-3d64-47c5-bd48-9bb7732617c1 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:16.022657 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-9b789493-3d64-47c5-bd48-9bb7732617c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:16.031113 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 704/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 14:43:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:16.083866 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c6cbc6-fc5c-4419-850a-054effda05f2 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:16.099631 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c6cbc6-fc5c-4419-850a-054effda05f2 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:16.105328 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0d2304-bb5c-460f-bde4-fae5036a0b5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0927c05c340c48e48b18754888aa9c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:16.105964 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0d2304-bb5c-460f-bde4-fae5036a0b5e service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:16.106245 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0d2304-bb5c-460f-bde4-fae5036a0b5e service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:16.106523 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0d2304-bb5c-460f-bde4-fae5036a0b5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:16.116664 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0d2304-bb5c-460f-bde4-fae5036a0b5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:16.118370 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0d2304-bb5c-460f-bde4-fae5036a0b5e service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:16.129461 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 712/1416] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:17.103849 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6882fa53-90d9-4df1-9503-016709efb146 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:17.104459 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6882fa53-90d9-4df1-9503-016709efb146 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:17.104459 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6882fa53-90d9-4df1-9503-016709efb146 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:17.105807 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 705/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 14:43:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:43:17.111008 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2331e84b-0a5c-40e0-99b3-52c0247a90c5 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:17.111454 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2331e84b-0a5c-40e0-99b3-52c0247a90c5 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:17.111572 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2331e84b-0a5c-40e0-99b3-52c0247a90c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:17.130313 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-2331e84b-0a5c-40e0-99b3-52c0247a90c5 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:17.131522 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-2331e84b-0a5c-40e0-99b3-52c0247a90c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:17.139602 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 713/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 14:43:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:43:17.308077 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6882fa53-90d9-4df1-9503-016709efb146 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:17.323797 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6882fa53-90d9-4df1-9503-016709efb146 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:17.330365 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92db68ed-6a26-440d-8207-87ca879cc2e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0927c05c340c48e48b18754888aa9c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:17.331019 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92db68ed-6a26-440d-8207-87ca879cc2e7 service nova] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:17.331307 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92db68ed-6a26-440d-8207-87ca879cc2e7 service nova] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:17.331584 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92db68ed-6a26-440d-8207-87ca879cc2e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:17.340373 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92db68ed-6a26-440d-8207-87ca879cc2e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:17.341946 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92db68ed-6a26-440d-8207-87ca879cc2e7 service nova] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:17.352528 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 706/1419] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:18.318169 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ec5f3-8f68-4b20-868f-b4ad1c51ff75 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:18.318720 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ec5f3-8f68-4b20-868f-b4ad1c51ff75 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:18.318720 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ec5f3-8f68-4b20-868f-b4ad1c51ff75 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:18.320030 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 714/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 14:43:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:43:18.325196 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0296bd0-591d-4ba4-ac60-25c360e4b072 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:18.325647 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0296bd0-591d-4ba4-ac60-25c360e4b072 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:18.325756 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0296bd0-591d-4ba4-ac60-25c360e4b072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:18.346940 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-a0296bd0-591d-4ba4-ac60-25c360e4b072 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:18.348217 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-a0296bd0-591d-4ba4-ac60-25c360e4b072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:18.355968 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 707/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 14:43:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:18.511418 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836ec5f3-8f68-4b20-868f-b4ad1c51ff75 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:18.527736 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836ec5f3-8f68-4b20-868f-b4ad1c51ff75 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:18.533481 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3183b8c-77b0-4a75-91b8-ee7402eab419 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0927c05c340c48e48b18754888aa9c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:18.534130 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3183b8c-77b0-4a75-91b8-ee7402eab419 service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:18.534413 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3183b8c-77b0-4a75-91b8-ee7402eab419 service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:18.534688 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3183b8c-77b0-4a75-91b8-ee7402eab419 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:18.543721 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3183b8c-77b0-4a75-91b8-ee7402eab419 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:18.545639 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3183b8c-77b0-4a75-91b8-ee7402eab419 service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:18.557487 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 715/1422] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:19.483597 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b7365-30d4-4c46-b784-1e2cc26663e1 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:19.484370 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b7365-30d4-4c46-b784-1e2cc26663e1 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:19.484370 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b7365-30d4-4c46-b784-1e2cc26663e1 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:19.485564 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 708/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 14:43:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:43:19.490520 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347efb5-2687-445b-95bc-9ff10fb64c2b None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:19.490941 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347efb5-2687-445b-95bc-9ff10fb64c2b None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:19.491105 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347efb5-2687-445b-95bc-9ff10fb64c2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:19.510985 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-d347efb5-2687-445b-95bc-9ff10fb64c2b None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:19.512263 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-d347efb5-2687-445b-95bc-9ff10fb64c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:19.521429 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 716/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 14:43:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:19.675972 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9b7365-30d4-4c46-b784-1e2cc26663e1 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:19.691220 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9b7365-30d4-4c46-b784-1e2cc26663e1 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:19.696646 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbeef62-52d3-4efa-8e81-3e8957a61159 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0927c05c340c48e48b18754888aa9c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:19.697291 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbeef62-52d3-4efa-8e81-3e8957a61159 service nova] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:19.697575 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbeef62-52d3-4efa-8e81-3e8957a61159 service nova] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:19.697853 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbeef62-52d3-4efa-8e81-3e8957a61159 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:19.706513 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbeef62-52d3-4efa-8e81-3e8957a61159 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:19.708130 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbeef62-52d3-4efa-8e81-3e8957a61159 service nova] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:19.717822 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 709/1425] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:20.662281 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f4e13-654b-4aae-82f1-f1d780bb34e1 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:20.662867 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f4e13-654b-4aae-82f1-f1d780bb34e1 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:20.662867 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f4e13-654b-4aae-82f1-f1d780bb34e1 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:20.664261 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 717/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 14:43:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:43:20.669360 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8a607e-70ab-42bf-bfa7-50e1e8db1e7d None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:20.669795 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8a607e-70ab-42bf-bfa7-50e1e8db1e7d None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:20.669956 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8a607e-70ab-42bf-bfa7-50e1e8db1e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:20.689152 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-0c8a607e-70ab-42bf-bfa7-50e1e8db1e7d None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:20.690363 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-0c8a607e-70ab-42bf-bfa7-50e1e8db1e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:20.698532 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 710/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 14:43:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:20.860806 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6f4e13-654b-4aae-82f1-f1d780bb34e1 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:20.876504 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6f4e13-654b-4aae-82f1-f1d780bb34e1 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:20.881908 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e0c4b1-cdb2-4da3-be53-fef01ea16bef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0927c05c340c48e48b18754888aa9c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:20.882536 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e0c4b1-cdb2-4da3-be53-fef01ea16bef service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:20.882817 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e0c4b1-cdb2-4da3-be53-fef01ea16bef service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:20.883093 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e0c4b1-cdb2-4da3-be53-fef01ea16bef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:20.892611 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e0c4b1-cdb2-4da3-be53-fef01ea16bef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:20.894189 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e0c4b1-cdb2-4da3-be53-fef01ea16bef service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:20.905179 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 718/1428] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:21.871806 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a7a70-42ce-4984-998b-c61e49c2aa4f None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:21.871806 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a7a70-42ce-4984-998b-c61e49c2aa4f None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:21.871806 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a7a70-42ce-4984-998b-c61e49c2aa4f None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:21.871806 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 711/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 14:43:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:43:21.877089 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb9adee-67dd-4504-98f6-2646cce40323 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:21.879691 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb9adee-67dd-4504-98f6-2646cce40323 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:21.879691 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb9adee-67dd-4504-98f6-2646cce40323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:21.896156 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-4eb9adee-67dd-4504-98f6-2646cce40323 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:21.897357 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-4eb9adee-67dd-4504-98f6-2646cce40323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:21.905220 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 719/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 14:43:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:22.058871 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9a7a70-42ce-4984-998b-c61e49c2aa4f None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:22.074505 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9a7a70-42ce-4984-998b-c61e49c2aa4f None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:22.080345 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0480a23c-17e2-4213-8ed4-e1ee77bb8bdf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0927c05c340c48e48b18754888aa9c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:22.080984 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0480a23c-17e2-4213-8ed4-e1ee77bb8bdf service nova] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:22.081265 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0480a23c-17e2-4213-8ed4-e1ee77bb8bdf service nova] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:22.081542 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0480a23c-17e2-4213-8ed4-e1ee77bb8bdf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:22.089840 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0480a23c-17e2-4213-8ed4-e1ee77bb8bdf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:22.091408 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0480a23c-17e2-4213-8ed4-e1ee77bb8bdf service nova] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:22.101429 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 712/1431] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:23.092988 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c13143-66e3-4975-afd0-e33bd33b0c60 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:23.094076 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c13143-66e3-4975-afd0-e33bd33b0c60 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:23.094076 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c13143-66e3-4975-afd0-e33bd33b0c60 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:23.095603 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 720/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 14:43:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:43:23.101056 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30ce0af-02a3-4a1e-be41-abca2bc9ec18 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:23.101619 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30ce0af-02a3-4a1e-be41-abca2bc9ec18 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:23.101619 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30ce0af-02a3-4a1e-be41-abca2bc9ec18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:23.120775 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-e30ce0af-02a3-4a1e-be41-abca2bc9ec18 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:23.122037 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-e30ce0af-02a3-4a1e-be41-abca2bc9ec18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:23.135164 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 713/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 14:43:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:23.296835 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c13143-66e3-4975-afd0-e33bd33b0c60 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:23.313938 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c13143-66e3-4975-afd0-e33bd33b0c60 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:23.320509 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29bbf7f-df21-4cc2-8632-80f09ae74811 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0927c05c340c48e48b18754888aa9c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:23.324855 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29bbf7f-df21-4cc2-8632-80f09ae74811 service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:23.325150 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29bbf7f-df21-4cc2-8632-80f09ae74811 service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:23.325433 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29bbf7f-df21-4cc2-8632-80f09ae74811 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:23.335828 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29bbf7f-df21-4cc2-8632-80f09ae74811 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:23.349629 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29bbf7f-df21-4cc2-8632-80f09ae74811 service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:23.349629 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 721/1434] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:24.364914 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527ce700-06fe-4453-a816-72f47577ab75 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:24.365456 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527ce700-06fe-4453-a816-72f47577ab75 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:24.365456 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527ce700-06fe-4453-a816-72f47577ab75 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:24.366788 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 714/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 14:43:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:43:24.372272 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a4d755-1907-4900-b6cf-5e9be986e4dd None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:24.372787 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a4d755-1907-4900-b6cf-5e9be986e4dd None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:24.372882 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a4d755-1907-4900-b6cf-5e9be986e4dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:24.392737 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-25a4d755-1907-4900-b6cf-5e9be986e4dd None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:24.393953 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-25a4d755-1907-4900-b6cf-5e9be986e4dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:24.402040 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 722/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 14:43:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:24.556945 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527ce700-06fe-4453-a816-72f47577ab75 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:24.573419 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527ce700-06fe-4453-a816-72f47577ab75 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:24.579774 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c90d70-cfa6-49ee-8f79-29359648b0a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0927c05c340c48e48b18754888aa9c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:24.580486 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c90d70-cfa6-49ee-8f79-29359648b0a5 service nova] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:24.580881 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c90d70-cfa6-49ee-8f79-29359648b0a5 service nova] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:24.581053 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c90d70-cfa6-49ee-8f79-29359648b0a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:24.590219 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c90d70-cfa6-49ee-8f79-29359648b0a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:24.591878 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c90d70-cfa6-49ee-8f79-29359648b0a5 service nova] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:24.603693 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 715/1437] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:25.558751 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b37d2c-bbe2-421c-a070-d14b953b2108 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:25.559334 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b37d2c-bbe2-421c-a070-d14b953b2108 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:25.559334 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b37d2c-bbe2-421c-a070-d14b953b2108 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:25.560762 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 723/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 14:43:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:43:25.571932 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662afd1e-7390-4d04-9bb2-433a7d121364 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:25.572550 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662afd1e-7390-4d04-9bb2-433a7d121364 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:25.572550 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662afd1e-7390-4d04-9bb2-433a7d121364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:25.593126 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-662afd1e-7390-4d04-9bb2-433a7d121364 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:25.594742 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-662afd1e-7390-4d04-9bb2-433a7d121364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:25.602921 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 716/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 14:43:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:25.764566 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b37d2c-bbe2-421c-a070-d14b953b2108 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:25.780511 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b37d2c-bbe2-421c-a070-d14b953b2108 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:25.786521 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6d0721-f62b-46f6-9244-8d44cb72040f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0927c05c340c48e48b18754888aa9c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:25.787191 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d0721-f62b-46f6-9244-8d44cb72040f service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:25.787478 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d0721-f62b-46f6-9244-8d44cb72040f service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:25.787755 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d0721-f62b-46f6-9244-8d44cb72040f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:25.798013 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6d0721-f62b-46f6-9244-8d44cb72040f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:25.799625 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6d0721-f62b-46f6-9244-8d44cb72040f service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:25.810987 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 724/1440] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:27.925379 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aefa3f-6f50-43bf-97b7-1c3a192a713b None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:27.925936 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aefa3f-6f50-43bf-97b7-1c3a192a713b None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:27.925936 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aefa3f-6f50-43bf-97b7-1c3a192a713b None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:27.927267 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 717/1441] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 14:43:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:43:27.933014 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4bb197-e98e-476d-9f23-a30285a93534 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:27.933619 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4bb197-e98e-476d-9f23-a30285a93534 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:27.933619 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4bb197-e98e-476d-9f23-a30285a93534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:27.953641 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-4b4bb197-e98e-476d-9f23-a30285a93534 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:27.955736 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-4b4bb197-e98e-476d-9f23-a30285a93534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:27.963001 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 725/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 14:43:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:28.002624 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aefa3f-6f50-43bf-97b7-1c3a192a713b None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:28.013916 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aefa3f-6f50-43bf-97b7-1c3a192a713b None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:28.018054 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68f5577-e7b0-4915-9b67-ed19dd90e0ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:28.018705 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f5577-e7b0-4915-9b67-ed19dd90e0ab service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:28.019015 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f5577-e7b0-4915-9b67-ed19dd90e0ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:28.019295 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f5577-e7b0-4915-9b67-ed19dd90e0ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:28.029206 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68f5577-e7b0-4915-9b67-ed19dd90e0ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:28.031824 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68f5577-e7b0-4915-9b67-ed19dd90e0ab service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:28.043170 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 718/1443] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:30.372902 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8407d9e-4c0d-42a4-970d-c1e72adae72d None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:30.373443 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8407d9e-4c0d-42a4-970d-c1e72adae72d None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:30.373561 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8407d9e-4c0d-42a4-970d-c1e72adae72d None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:30.374939 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 726/1444] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 14:43:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:43:30.380983 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc321f4-65e5-41f8-88bf-515cd5b889d2 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:30.381519 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc321f4-65e5-41f8-88bf-515cd5b889d2 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:30.381519 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc321f4-65e5-41f8-88bf-515cd5b889d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:30.401351 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-4cc321f4-65e5-41f8-88bf-515cd5b889d2 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:30.402563 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-4cc321f4-65e5-41f8-88bf-515cd5b889d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:30.411422 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 719/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 14:43:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:30.449839 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8407d9e-4c0d-42a4-970d-c1e72adae72d None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:30.461498 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8407d9e-4c0d-42a4-970d-c1e72adae72d None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:30.464994 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4524863-6f0d-4878-a4b7-284656b9a91e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:30.465642 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4524863-6f0d-4878-a4b7-284656b9a91e service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:30.465927 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4524863-6f0d-4878-a4b7-284656b9a91e service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:30.466208 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4524863-6f0d-4878-a4b7-284656b9a91e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:30.476767 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4524863-6f0d-4878-a4b7-284656b9a91e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:30.478223 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4524863-6f0d-4878-a4b7-284656b9a91e service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:30.488455 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 727/1446] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:33.062031 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8297418d-0d9d-4e18-885f-00cac6492f0b None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:33.062616 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8297418d-0d9d-4e18-885f-00cac6492f0b None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:33.062742 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8297418d-0d9d-4e18-885f-00cac6492f0b None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:33.064038 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 720/1447] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 14:43:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:43:33.069747 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c2075-2787-4ce6-9541-5be8ff5932a4 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:33.070481 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c2075-2787-4ce6-9541-5be8ff5932a4 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:33.070481 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c2075-2787-4ce6-9541-5be8ff5932a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:33.088876 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-537c2075-2787-4ce6-9541-5be8ff5932a4 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:33.090106 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-537c2075-2787-4ce6-9541-5be8ff5932a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:33.098623 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 728/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 14:43:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:33.138535 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8297418d-0d9d-4e18-885f-00cac6492f0b None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:33.140996 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-8297418d-0d9d-4e18-885f-00cac6492f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:33.156379 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8297418d-0d9d-4e18-885f-00cac6492f0b None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:33.160430 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3759c8ab-5e46-4cb3-8e8b-4d5dd5b9f0ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:33.161083 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759c8ab-5e46-4cb3-8e8b-4d5dd5b9f0ec service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:33.161370 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759c8ab-5e46-4cb3-8e8b-4d5dd5b9f0ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:33.161652 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759c8ab-5e46-4cb3-8e8b-4d5dd5b9f0ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:33.171243 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3759c8ab-5e46-4cb3-8e8b-4d5dd5b9f0ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:33.172750 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3759c8ab-5e46-4cb3-8e8b-4d5dd5b9f0ec service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:33.182362 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 721/1449] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:35.630297 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d96c8-82fb-4ee2-b369-d25cb57d22df None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:35.630874 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d96c8-82fb-4ee2-b369-d25cb57d22df None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:35.630874 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d96c8-82fb-4ee2-b369-d25cb57d22df None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:35.632254 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 729/1450] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 14:43:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:43:35.638005 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e582d3e-d4dd-4064-a81e-706e9b9f79f7 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:35.638747 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e582d3e-d4dd-4064-a81e-706e9b9f79f7 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:35.638867 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e582d3e-d4dd-4064-a81e-706e9b9f79f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:35.656826 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-6e582d3e-d4dd-4064-a81e-706e9b9f79f7 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:35.658032 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-6e582d3e-d4dd-4064-a81e-706e9b9f79f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:35.665951 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 722/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 14:43:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:35.705580 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4d96c8-82fb-4ee2-b369-d25cb57d22df None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:35.716415 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4d96c8-82fb-4ee2-b369-d25cb57d22df None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:35.720615 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f1386b-b165-469c-8325-82eec9180a10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:35.721262 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f1386b-b165-469c-8325-82eec9180a10 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:35.721550 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f1386b-b165-469c-8325-82eec9180a10 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:35.721825 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f1386b-b165-469c-8325-82eec9180a10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:35.731396 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f1386b-b165-469c-8325-82eec9180a10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:35.732891 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f1386b-b165-469c-8325-82eec9180a10 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:35.743759 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 730/1452] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:37.966799 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5470a26b-6322-46f5-a7da-2466acdbbb4e None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:37.968003 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5470a26b-6322-46f5-a7da-2466acdbbb4e None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:37.968236 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5470a26b-6322-46f5-a7da-2466acdbbb4e None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:37.969722 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 723/1453] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 14:43:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:43:37.975696 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044828f4-c940-4a45-a834-0d68a9d3705c None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:37.976253 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044828f4-c940-4a45-a834-0d68a9d3705c None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:37.976566 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044828f4-c940-4a45-a834-0d68a9d3705c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:37.994981 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-044828f4-c940-4a45-a834-0d68a9d3705c None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:37.996236 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-044828f4-c940-4a45-a834-0d68a9d3705c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:38.004688 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 731/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 14:43:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:38.045506 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5470a26b-6322-46f5-a7da-2466acdbbb4e None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:38.056944 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5470a26b-6322-46f5-a7da-2466acdbbb4e None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:38.060513 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173833b9-17fe-424a-a85c-27cab798ef45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:38.061647 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173833b9-17fe-424a-a85c-27cab798ef45 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:38.062077 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173833b9-17fe-424a-a85c-27cab798ef45 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:38.062578 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173833b9-17fe-424a-a85c-27cab798ef45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:38.072285 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173833b9-17fe-424a-a85c-27cab798ef45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:38.073727 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173833b9-17fe-424a-a85c-27cab798ef45 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:38.084841 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 724/1455] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:40.355846 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136fd05a-650d-41f9-b2e6-4d1102552aeb None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:40.356404 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136fd05a-650d-41f9-b2e6-4d1102552aeb None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:40.356404 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136fd05a-650d-41f9-b2e6-4d1102552aeb None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:40.357771 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 732/1456] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 14:43:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:43:40.363949 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb35700-1215-431c-a5b9-9336a37f433e None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:40.364458 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb35700-1215-431c-a5b9-9336a37f433e None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:40.364557 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb35700-1215-431c-a5b9-9336a37f433e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:40.384082 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-dbb35700-1215-431c-a5b9-9336a37f433e None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:40.385310 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-dbb35700-1215-431c-a5b9-9336a37f433e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:40.394993 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 725/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 14:43:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:40.433729 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136fd05a-650d-41f9-b2e6-4d1102552aeb None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:40.445179 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136fd05a-650d-41f9-b2e6-4d1102552aeb None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:40.448733 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63027313-0ede-4559-afb0-52c95f63a6f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:40.449369 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63027313-0ede-4559-afb0-52c95f63a6f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:40.449648 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63027313-0ede-4559-afb0-52c95f63a6f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:40.449924 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63027313-0ede-4559-afb0-52c95f63a6f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:40.460018 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63027313-0ede-4559-afb0-52c95f63a6f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:40.461474 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63027313-0ede-4559-afb0-52c95f63a6f7 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:40.472486 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 733/1458] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:42.749488 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43175629-3e3c-4cf1-af80-54ebadafb5c4 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:42.750185 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43175629-3e3c-4cf1-af80-54ebadafb5c4 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:42.750185 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43175629-3e3c-4cf1-af80-54ebadafb5c4 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:42.751441 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 726/1459] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 14:43:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:43:42.757104 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c7471-bd5a-44cf-ad19-757a5f67c3b0 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:42.757587 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c7471-bd5a-44cf-ad19-757a5f67c3b0 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:42.757714 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c7471-bd5a-44cf-ad19-757a5f67c3b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:42.764027 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=70 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:42.776862 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-9e4c7471-bd5a-44cf-ad19-757a5f67c3b0 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:42.779142 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-9e4c7471-bd5a-44cf-ad19-757a5f67c3b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:42.786618 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 734/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 14:43:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:42.833941 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43175629-3e3c-4cf1-af80-54ebadafb5c4 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:42.839372 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=73 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:42.844993 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43175629-3e3c-4cf1-af80-54ebadafb5c4 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:42.848857 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4c3a56-ccbc-4a18-9d83-5ff274003ce4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:42.849505 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4c3a56-ccbc-4a18-9d83-5ff274003ce4 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:42.849823 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4c3a56-ccbc-4a18-9d83-5ff274003ce4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:42.850134 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4c3a56-ccbc-4a18-9d83-5ff274003ce4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:42.860582 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4c3a56-ccbc-4a18-9d83-5ff274003ce4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:42.862014 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4c3a56-ccbc-4a18-9d83-5ff274003ce4 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:42.872245 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 727/1461] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:45.097634 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f7886-fa6f-4b7b-8e32-d4f1cc9b9072 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:45.098192 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f7886-fa6f-4b7b-8e32-d4f1cc9b9072 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:45.098192 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f7886-fa6f-4b7b-8e32-d4f1cc9b9072 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:45.099611 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 735/1462] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 14:43:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:43:45.105497 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40e1f95-e3c9-485c-994a-32f42ef4c7ba None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:45.106026 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40e1f95-e3c9-485c-994a-32f42ef4c7ba None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:45.106026 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40e1f95-e3c9-485c-994a-32f42ef4c7ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:45.125672 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-c40e1f95-e3c9-485c-994a-32f42ef4c7ba None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:45.126900 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-c40e1f95-e3c9-485c-994a-32f42ef4c7ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:45.135116 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 728/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 14:43:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:45.174130 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687f7886-fa6f-4b7b-8e32-d4f1cc9b9072 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:45.185828 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687f7886-fa6f-4b7b-8e32-d4f1cc9b9072 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:45.189744 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9ff3c2-fdac-4b9e-a512-64ebc41124e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:45.190384 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ff3c2-fdac-4b9e-a512-64ebc41124e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:45.190744 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ff3c2-fdac-4b9e-a512-64ebc41124e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:45.191023 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ff3c2-fdac-4b9e-a512-64ebc41124e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:45.200739 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9ff3c2-fdac-4b9e-a512-64ebc41124e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:45.202873 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9ff3c2-fdac-4b9e-a512-64ebc41124e4 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:45.213037 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 736/1464] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:47.436974 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b4435e-f37f-4c90-b969-3e7c415fe477 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:47.437533 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b4435e-f37f-4c90-b969-3e7c415fe477 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:47.437825 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b4435e-f37f-4c90-b969-3e7c415fe477 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:47.440152 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 729/1465] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 14:43:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:43:47.454272 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54ece6f-607a-4868-9b26-18f332aafa78 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:47.454856 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54ece6f-607a-4868-9b26-18f332aafa78 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:47.454856 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54ece6f-607a-4868-9b26-18f332aafa78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:47.476166 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-e54ece6f-607a-4868-9b26-18f332aafa78 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:47.477664 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e54ece6f-607a-4868-9b26-18f332aafa78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:47.485932 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 737/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 14:43:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:47.523898 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b4435e-f37f-4c90-b969-3e7c415fe477 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:47.535599 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b4435e-f37f-4c90-b969-3e7c415fe477 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:47.539173 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533d745c-6aab-4c29-b796-4df6f87f746a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:47.539819 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d745c-6aab-4c29-b796-4df6f87f746a service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:47.540127 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d745c-6aab-4c29-b796-4df6f87f746a service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:47.540410 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d745c-6aab-4c29-b796-4df6f87f746a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:47.549865 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533d745c-6aab-4c29-b796-4df6f87f746a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:47.551344 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533d745c-6aab-4c29-b796-4df6f87f746a service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:47.562107 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 730/1467] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:49.743181 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f608d5-f2ed-4cee-9939-e3b496e3fe2c None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:49.743715 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f608d5-f2ed-4cee-9939-e3b496e3fe2c None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:49.743828 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f608d5-f2ed-4cee-9939-e3b496e3fe2c None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:49.745232 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 738/1468] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 14:43:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:43:49.751682 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16abb254-5a44-4e6b-b4db-8c3e322be995 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:49.752201 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16abb254-5a44-4e6b-b4db-8c3e322be995 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:49.752307 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16abb254-5a44-4e6b-b4db-8c3e322be995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:49.770411 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-16abb254-5a44-4e6b-b4db-8c3e322be995 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:49.771660 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-16abb254-5a44-4e6b-b4db-8c3e322be995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:49.779954 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 731/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 14:43:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:49.819973 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f608d5-f2ed-4cee-9939-e3b496e3fe2c None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:49.831681 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f608d5-f2ed-4cee-9939-e3b496e3fe2c None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:49.835168 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41f42e7-6dcf-428d-99a2-47a22c69ff98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:49.835806 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41f42e7-6dcf-428d-99a2-47a22c69ff98 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:49.836082 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41f42e7-6dcf-428d-99a2-47a22c69ff98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:49.836386 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41f42e7-6dcf-428d-99a2-47a22c69ff98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:49.846827 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41f42e7-6dcf-428d-99a2-47a22c69ff98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:49.848508 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41f42e7-6dcf-428d-99a2-47a22c69ff98 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:49.859443 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 739/1470] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:59.765111 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=17 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:59.854325 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=17 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:10.825185 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdcee41-f543-4d28-b5e3-ff38ee8d818c None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:10.826477 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdcee41-f543-4d28-b5e3-ff38ee8d818c None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:10.826477 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdcee41-f543-4d28-b5e3-ff38ee8d818c None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:10.827765 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 732/1471] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:44:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:44:10.836512 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1b018e-49f7-4c77-b0bc-c5082e3df4e6 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:10.837089 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1b018e-49f7-4c77-b0bc-c5082e3df4e6 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:10.837250 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1b018e-49f7-4c77-b0bc-c5082e3df4e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:10.858761 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-8d1b018e-49f7-4c77-b0bc-c5082e3df4e6 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:10.859998 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-8d1b018e-49f7-4c77-b0bc-c5082e3df4e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:10.868210 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 740/1472] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:44:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:10.920352 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7164f1c9-0135-4660-8c0d-fa19e358dd7e None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:10.920637 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7164f1c9-0135-4660-8c0d-fa19e358dd7e None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:10.920925 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7164f1c9-0135-4660-8c0d-fa19e358dd7e None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:10.922307 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 733/1473] 10.222.0.42 () {54 vars in 962 bytes} [Wed Jun 15 14:44:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:44:10.927604 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e79d5d-ef42-48a1-8724-eb86ced0e5a3 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:10.927927 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e79d5d-ef42-48a1-8724-eb86ced0e5a3 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:10.928253 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e79d5d-ef42-48a1-8724-eb86ced0e5a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:10.947903 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-75e79d5d-ef42-48a1-8724-eb86ced0e5a3 None None] MFA Rules not processed for user `4f18c57f27784694935bfb0bcea48a78`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:10.949130 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-75e79d5d-ef42-48a1-8724-eb86ced0e5a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:10.960809 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 741/1474] 10.222.0.42 () {58 vars in 1059 bytes} [Wed Jun 15 14:44:10 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:44:10.969242 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7164f1c9-0135-4660-8c0d-fa19e358dd7e None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:10.980458 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7164f1c9-0135-4660-8c0d-fa19e358dd7e None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:10.984688 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47eaddb-5f49-42f3-8637-5f1d30e9bb0f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f18c57f27784694935bfb0bcea48a78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:10.985328 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47eaddb-5f49-42f3-8637-5f1d30e9bb0f service glance] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:10.985611 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47eaddb-5f49-42f3-8637-5f1d30e9bb0f service glance] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:10.985889 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47eaddb-5f49-42f3-8637-5f1d30e9bb0f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:10.996067 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47eaddb-5f49-42f3-8637-5f1d30e9bb0f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:11.004518 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47eaddb-5f49-42f3-8637-5f1d30e9bb0f service glance] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:11.008816 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 734/1475] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:44:13.362933 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d391a3d-70a8-4c2f-8638-b399013fd44a None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:13.363645 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d391a3d-70a8-4c2f-8638-b399013fd44a None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:13.363645 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d391a3d-70a8-4c2f-8638-b399013fd44a None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:13.365076 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 742/1476] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:44:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:44:13.371621 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a063c9-498a-4d43-af84-871169f177e4 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:13.373949 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a063c9-498a-4d43-af84-871169f177e4 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:13.373949 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a063c9-498a-4d43-af84-871169f177e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:13.391938 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-86a063c9-498a-4d43-af84-871169f177e4 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:13.393197 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-86a063c9-498a-4d43-af84-871169f177e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:13.401352 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 735/1477] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:44:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:13.426932 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d391a3d-70a8-4c2f-8638-b399013fd44a None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:13.443771 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d391a3d-70a8-4c2f-8638-b399013fd44a None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:13.451787 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a224a2-0e91-4ab2-9fc2-56e8de1289ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:13.453074 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a224a2-0e91-4ab2-9fc2-56e8de1289ca admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:13.453540 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a224a2-0e91-4ab2-9fc2-56e8de1289ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:13.453993 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a224a2-0e91-4ab2-9fc2-56e8de1289ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:13.455253 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a224a2-0e91-4ab2-9fc2-56e8de1289ca admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:13.458403 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: WARNING py.warnings [None req-c7a224a2-0e91-4ab2-9fc2-56e8de1289ca admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 14:44:13.458403 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: warnings.warn(msg) Jun 15 14:44:13.458403 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: Jun 15 14:44:13.460487 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a224a2-0e91-4ab2-9fc2-56e8de1289ca admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:13.493444 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 743/1478] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jun 15 14:44:13 2022] GET /identity/v3/projects => generated 2553 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:44:15.803969 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a1f81-9005-4c98-96d4-3adf277c1aa4 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:15.804536 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a1f81-9005-4c98-96d4-3adf277c1aa4 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:15.804536 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a1f81-9005-4c98-96d4-3adf277c1aa4 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:15.805907 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 736/1479] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:44:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:44:15.812251 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa58a47-eae2-451e-a8d8-87ae1fcea27c None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:15.812755 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa58a47-eae2-451e-a8d8-87ae1fcea27c None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:15.812952 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa58a47-eae2-451e-a8d8-87ae1fcea27c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:15.838148 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-7fa58a47-eae2-451e-a8d8-87ae1fcea27c None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:15.839829 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-7fa58a47-eae2-451e-a8d8-87ae1fcea27c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:15.849785 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 744/1480] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:44:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:15.910604 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09a1f81-9005-4c98-96d4-3adf277c1aa4 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:15.927290 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09a1f81-9005-4c98-96d4-3adf277c1aa4 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:15.933584 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53660d49-e856-4d40-8097-5d672eb8cd69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0927c05c340c48e48b18754888aa9c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:15.934330 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53660d49-e856-4d40-8097-5d672eb8cd69 service nova] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:15.934613 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53660d49-e856-4d40-8097-5d672eb8cd69 service nova] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:15.934888 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53660d49-e856-4d40-8097-5d672eb8cd69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:15.944709 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53660d49-e856-4d40-8097-5d672eb8cd69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:15.946351 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53660d49-e856-4d40-8097-5d672eb8cd69 service nova] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:15.958158 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 737/1481] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:44:18.528366 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dde03b-1a9f-48c6-abb9-30b90a7b20b9 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:18.529577 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dde03b-1a9f-48c6-abb9-30b90a7b20b9 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:18.529577 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dde03b-1a9f-48c6-abb9-30b90a7b20b9 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:18.531054 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 745/1482] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:44:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:44:18.538265 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac284d-814b-42bf-afa0-076d6c603b3b None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:18.538957 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac284d-814b-42bf-afa0-076d6c603b3b None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:18.538957 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac284d-814b-42bf-afa0-076d6c603b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:18.559609 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-27ac284d-814b-42bf-afa0-076d6c603b3b None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:18.561591 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-27ac284d-814b-42bf-afa0-076d6c603b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:18.570201 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 738/1483] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:44:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:44:18.610127 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dde03b-1a9f-48c6-abb9-30b90a7b20b9 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:18.622059 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dde03b-1a9f-48c6-abb9-30b90a7b20b9 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:18.628132 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feee21a-7aa8-44a2-bbfe-a5cfc1ca2e65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:18.629272 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feee21a-7aa8-44a2-bbfe-a5cfc1ca2e65 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:18.629759 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feee21a-7aa8-44a2-bbfe-a5cfc1ca2e65 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:18.630243 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feee21a-7aa8-44a2-bbfe-a5cfc1ca2e65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:18.646041 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feee21a-7aa8-44a2-bbfe-a5cfc1ca2e65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:18.647619 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feee21a-7aa8-44a2-bbfe-a5cfc1ca2e65 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:18.661694 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 746/1484] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:44:21.262413 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1a32cf-0d56-4476-bd8c-539cf149fdf8 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:21.263791 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1a32cf-0d56-4476-bd8c-539cf149fdf8 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:21.263791 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1a32cf-0d56-4476-bd8c-539cf149fdf8 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:21.265786 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 739/1485] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:44:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:44:21.271869 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3730aeb8-c14f-465b-9b14-adbcd215f0a5 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:21.272487 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3730aeb8-c14f-465b-9b14-adbcd215f0a5 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:21.272487 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3730aeb8-c14f-465b-9b14-adbcd215f0a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:21.296742 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-3730aeb8-c14f-465b-9b14-adbcd215f0a5 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:21.296742 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-3730aeb8-c14f-465b-9b14-adbcd215f0a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:21.303531 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 747/1486] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:44:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:44:21.343007 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1a32cf-0d56-4476-bd8c-539cf149fdf8 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:21.361076 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1a32cf-0d56-4476-bd8c-539cf149fdf8 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:21.367184 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed2b15e-772b-4bf7-b29a-3b60782604cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:21.368285 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed2b15e-772b-4bf7-b29a-3b60782604cd service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:21.368767 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed2b15e-772b-4bf7-b29a-3b60782604cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:21.369223 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed2b15e-772b-4bf7-b29a-3b60782604cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:21.386521 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed2b15e-772b-4bf7-b29a-3b60782604cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:21.390051 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed2b15e-772b-4bf7-b29a-3b60782604cd service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:21.408975 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 740/1487] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:44:23.927219 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742961a2-33ed-4019-9a07-dac92d7aa4b0 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:23.927785 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742961a2-33ed-4019-9a07-dac92d7aa4b0 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:23.927982 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742961a2-33ed-4019-9a07-dac92d7aa4b0 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:23.929880 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 748/1488] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:44:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:44:23.937653 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc17c937-768b-41fc-be2d-a37e7371c845 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:23.938489 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc17c937-768b-41fc-be2d-a37e7371c845 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:23.938489 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc17c937-768b-41fc-be2d-a37e7371c845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:23.960234 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-bc17c937-768b-41fc-be2d-a37e7371c845 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:23.961455 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-bc17c937-768b-41fc-be2d-a37e7371c845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:23.970017 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 741/1489] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:44:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:24.023800 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742961a2-33ed-4019-9a07-dac92d7aa4b0 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:24.035698 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742961a2-33ed-4019-9a07-dac92d7aa4b0 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:24.039752 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a798e673-a8cb-4600-b30e-f4e21d24aea5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:24.040457 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a798e673-a8cb-4600-b30e-f4e21d24aea5 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:24.040728 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a798e673-a8cb-4600-b30e-f4e21d24aea5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:24.041003 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a798e673-a8cb-4600-b30e-f4e21d24aea5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:24.052616 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a798e673-a8cb-4600-b30e-f4e21d24aea5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:24.054063 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a798e673-a8cb-4600-b30e-f4e21d24aea5 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:24.065144 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 749/1490] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:44:26.847094 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d353ad49-b625-4ef1-9c6c-491f3cc6b3d2 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:26.847713 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d353ad49-b625-4ef1-9c6c-491f3cc6b3d2 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:26.847713 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d353ad49-b625-4ef1-9c6c-491f3cc6b3d2 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:26.849105 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 742/1491] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jun 15 14:44:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:44:26.855448 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a736064c-5195-4b5c-bdb4-898375194c2b None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:26.856075 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a736064c-5195-4b5c-bdb4-898375194c2b None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:26.856075 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a736064c-5195-4b5c-bdb4-898375194c2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:26.875321 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-a736064c-5195-4b5c-bdb4-898375194c2b None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:26.876596 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a736064c-5195-4b5c-bdb4-898375194c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:26.885429 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 750/1492] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jun 15 14:44:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:26.928844 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d353ad49-b625-4ef1-9c6c-491f3cc6b3d2 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:26.939999 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d353ad49-b625-4ef1-9c6c-491f3cc6b3d2 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:26.943869 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56abe66a-2a5c-4928-a73b-af5b50432895 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:26.944560 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56abe66a-2a5c-4928-a73b-af5b50432895 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:26.944833 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56abe66a-2a5c-4928-a73b-af5b50432895 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:26.945116 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56abe66a-2a5c-4928-a73b-af5b50432895 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:26.955611 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56abe66a-2a5c-4928-a73b-af5b50432895 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:26.957093 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56abe66a-2a5c-4928-a73b-af5b50432895 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:26.968249 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 743/1493] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:44:36.869768 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=33 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:36.963039 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=33 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:46:13.278400 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e17352-02d2-436c-ba62-f55d7d2bb803 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:46:13.279022 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e17352-02d2-436c-ba62-f55d7d2bb803 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:46:13.279022 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e17352-02d2-436c-ba62-f55d7d2bb803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:46:13.297889 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-54e17352-02d2-436c-ba62-f55d7d2bb803 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:46:13.299117 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-54e17352-02d2-436c-ba62-f55d7d2bb803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:46:13.307666 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 751/1494] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:46:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:46:13.317977 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd78497e-068e-4c41-a963-718554bedc67 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:46:13.318549 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd78497e-068e-4c41-a963-718554bedc67 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:46:13.318549 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd78497e-068e-4c41-a963-718554bedc67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:46:13.337638 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-fd78497e-068e-4c41-a963-718554bedc67 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:46:13.353946 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-fd78497e-068e-4c41-a963-718554bedc67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:46:13.353946 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 744/1495] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:46:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:46:13.354227 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e17352-02d2-436c-ba62-f55d7d2bb803 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:46:13.364602 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e17352-02d2-436c-ba62-f55d7d2bb803 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:46:13.370224 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049e499e-3f1f-4cba-9811-eb26bdff5233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:46:13.371225 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e499e-3f1f-4cba-9811-eb26bdff5233 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:46:13.371536 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e499e-3f1f-4cba-9811-eb26bdff5233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:46:13.371812 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e499e-3f1f-4cba-9811-eb26bdff5233 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:46:13.372606 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049e499e-3f1f-4cba-9811-eb26bdff5233 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:46:13.372919 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049e499e-3f1f-4cba-9811-eb26bdff5233 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:46:13.373941 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: WARNING py.warnings [None req-049e499e-3f1f-4cba-9811-eb26bdff5233 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 15 14:46:13.373941 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: warnings.warn(msg) Jun 15 14:46:13.373941 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: Jun 15 14:46:13.375002 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049e499e-3f1f-4cba-9811-eb26bdff5233 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:46:13.389799 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 752/1496] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jun 15 14:46:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:46:13.398544 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd78497e-068e-4c41-a963-718554bedc67 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:46:13.409190 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd78497e-068e-4c41-a963-718554bedc67 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:46:13.413834 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde4e9c4-4764-4a78-a036-41cbe9019ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:46:13.414494 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde4e9c4-4764-4a78-a036-41cbe9019ae7 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:46:13.414778 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde4e9c4-4764-4a78-a036-41cbe9019ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:46:13.415057 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde4e9c4-4764-4a78-a036-41cbe9019ae7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:46:13.415765 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde4e9c4-4764-4a78-a036-41cbe9019ae7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:46:13.418257 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: WARNING py.warnings [None req-cde4e9c4-4764-4a78-a036-41cbe9019ae7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 14:46:13.418257 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: warnings.warn(msg) Jun 15 14:46:13.418257 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: Jun 15 14:46:13.419460 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde4e9c4-4764-4a78-a036-41cbe9019ae7 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:46:13.561400 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 745/1497] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:46:13 2022] POST /identity/v3/projects => generated 377 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:46:13.566169 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049e499e-3f1f-4cba-9811-eb26bdff5233 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:46:13.578428 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049e499e-3f1f-4cba-9811-eb26bdff5233 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:46:13.657136 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed97b3c-7bf0-4ccb-b1ab-1bd8b1c46d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:46:13.657885 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed97b3c-7bf0-4ccb-b1ab-1bd8b1c46d12 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:46:13.658172 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed97b3c-7bf0-4ccb-b1ab-1bd8b1c46d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:46:13.658452 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed97b3c-7bf0-4ccb-b1ab-1bd8b1c46d12 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:46:13.659153 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed97b3c-7bf0-4ccb-b1ab-1bd8b1c46d12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:46:13.660299 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: WARNING py.warnings [None req-0ed97b3c-7bf0-4ccb-b1ab-1bd8b1c46d12 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 14:46:13.660299 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: warnings.warn(msg) Jun 15 14:46:13.660299 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: Jun 15 14:46:13.661573 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed97b3c-7bf0-4ccb-b1ab-1bd8b1c46d12 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:46:13.798914 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 753/1498] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:46:13 2022] POST /identity/v3/users => generated 289 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:46:13.803228 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde4e9c4-4764-4a78-a036-41cbe9019ae7 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:46:13.818931 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde4e9c4-4764-4a78-a036-41cbe9019ae7 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:46:13.818931 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd14d53-ff9f-4388-b35d-13543fe0cc69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:46:13.819565 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd14d53-ff9f-4388-b35d-13543fe0cc69 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:46:13.819927 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd14d53-ff9f-4388-b35d-13543fe0cc69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:46:13.820153 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd14d53-ff9f-4388-b35d-13543fe0cc69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:46:13.820891 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd14d53-ff9f-4388-b35d-13543fe0cc69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:46:13.821923 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: WARNING py.warnings [None req-7bd14d53-ff9f-4388-b35d-13543fe0cc69 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 15 14:46:13.821923 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: warnings.warn(msg) Jun 15 14:46:13.821923 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: Jun 15 14:46:13.822811 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd14d53-ff9f-4388-b35d-13543fe0cc69 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:46:13.837299 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 746/1499] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:46:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:46:13.842428 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed97b3c-7bf0-4ccb-b1ab-1bd8b1c46d12 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:46:13.853553 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed97b3c-7bf0-4ccb-b1ab-1bd8b1c46d12 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:46:13.858352 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26528265-d8cd-449a-a23a-f50c0d00f5e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:46:13.859111 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26528265-d8cd-449a-a23a-f50c0d00f5e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:46:13.859398 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26528265-d8cd-449a-a23a-f50c0d00f5e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:46:13.859704 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26528265-d8cd-449a-a23a-f50c0d00f5e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:46:13.860483 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26528265-d8cd-449a-a23a-f50c0d00f5e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:46:13.861495 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: WARNING py.warnings [None req-26528265-d8cd-449a-a23a-f50c0d00f5e7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 15 14:46:13.861495 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: warnings.warn(msg) Jun 15 14:46:13.861495 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: Jun 15 14:46:13.862386 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26528265-d8cd-449a-a23a-f50c0d00f5e7 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:46:13.876841 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 754/1500] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:46:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:46:13.881289 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd14d53-ff9f-4388-b35d-13543fe0cc69 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:46:13.892855 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd14d53-ff9f-4388-b35d-13543fe0cc69 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:46:13.897553 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d2a608-1da4-449b-9211-6491b554a592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:46:13.910401 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d2a608-1da4-449b-9211-6491b554a592 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:46:13.910401 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d2a608-1da4-449b-9211-6491b554a592 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:46:13.910401 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d2a608-1da4-449b-9211-6491b554a592 admin admin] PATH_INFO: `/v3/projects/86a6a53d321f46a0bf8902180dc4ca46/users/f7552545631e4947b34189a087b5457c/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:46:13.915623 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d2a608-1da4-449b-9211-6491b554a592 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86a6a53d321f46a0bf8902180dc4ca46, user_id=f7552545631e4947b34189a087b5457c, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:46:13.917567 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: WARNING py.warnings [None req-60d2a608-1da4-449b-9211-6491b554a592 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 14:46:13.917567 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: warnings.warn(msg) Jun 15 14:46:13.917567 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: Jun 15 14:46:13.919485 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d2a608-1da4-449b-9211-6491b554a592 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:46:14.032592 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 747/1501] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:46:13 2022] PUT /identity/v3/projects/86a6a53d321f46a0bf8902180dc4ca46/users/f7552545631e4947b34189a087b5457c/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:46:14.043999 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a47492-be29-4709-877d-31a3bff622bc None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:46:14.044341 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a47492-be29-4709-877d-31a3bff622bc None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:46:14.044603 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a47492-be29-4709-877d-31a3bff622bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:46:14.074395 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-16a47492-be29-4709-877d-31a3bff622bc None None] MFA Rules not processed for user `f7552545631e4947b34189a087b5457c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:46:14.075582 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-16a47492-be29-4709-877d-31a3bff622bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:46:14.145375 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 755/1502] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:46:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:46:14.159427 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ff713e-42fb-4a34-af8b-008dbfc00791 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:46:14.159751 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ff713e-42fb-4a34-af8b-008dbfc00791 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:46:14.160040 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ff713e-42fb-4a34-af8b-008dbfc00791 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:46:14.161678 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 748/1503] 10.222.0.42 () {54 vars in 963 bytes} [Wed Jun 15 14:46:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:46:14.166783 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407525bf-c890-4942-b299-685e700ae271 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:46:14.167202 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407525bf-c890-4942-b299-685e700ae271 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:46:14.167749 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407525bf-c890-4942-b299-685e700ae271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:46:14.186851 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-407525bf-c890-4942-b299-685e700ae271 None None] MFA Rules not processed for user `e44f6cd435be4e03bc79cb3125d71451`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:46:14.188206 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-407525bf-c890-4942-b299-685e700ae271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:46:14.236241 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:46:14.243846 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 756/1504] 10.222.0.42 () {58 vars in 1060 bytes} [Wed Jun 15 14:46:14 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:46:14.251983 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ff713e-42fb-4a34-af8b-008dbfc00791 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:46:14.264063 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ff713e-42fb-4a34-af8b-008dbfc00791 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:46:14.267728 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306a5291-a1e0-4b1e-8260-faba2fa25a9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:46:14.268424 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306a5291-a1e0-4b1e-8260-faba2fa25a9d service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:46:14.268732 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306a5291-a1e0-4b1e-8260-faba2fa25a9d service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:46:14.269015 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306a5291-a1e0-4b1e-8260-faba2fa25a9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:46:14.279481 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306a5291-a1e0-4b1e-8260-faba2fa25a9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:46:14.281072 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306a5291-a1e0-4b1e-8260-faba2fa25a9d service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:46:14.290967 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 749/1505] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:46:14 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:46:14.335622 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fc5e6d-eff9-4efa-8b14-73b7f7af7a87 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:46:14.336431 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fc5e6d-eff9-4efa-8b14-73b7f7af7a87 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:46:14.336622 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fc5e6d-eff9-4efa-8b14-73b7f7af7a87 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:46:14.338310 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 757/1506] 10.222.0.42 () {52 vars in 829 bytes} [Wed Jun 15 14:46:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:46:14.342746 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306a5291-a1e0-4b1e-8260-faba2fa25a9d service cinder] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:46:14.410016 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306a5291-a1e0-4b1e-8260-faba2fa25a9d service cinder] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:46:14.414950 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d703e2c1-eb94-4963-9a5b-03e9b6524ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:46:14.415841 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d703e2c1-eb94-4963-9a5b-03e9b6524ac2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:46:14.416142 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d703e2c1-eb94-4963-9a5b-03e9b6524ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:46:14.416426 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d703e2c1-eb94-4963-9a5b-03e9b6524ac2 admin admin] PATH_INFO: `/v3/users/f7552545631e4947b34189a087b5457c` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:46:14.418115 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d703e2c1-eb94-4963-9a5b-03e9b6524ac2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7552545631e4947b34189a087b5457c)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:46:14.419595 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: WARNING py.warnings [None req-d703e2c1-eb94-4963-9a5b-03e9b6524ac2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 14:46:14.419595 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: warnings.warn(msg) Jun 15 14:46:14.419595 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: Jun 15 14:46:14.421018 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d703e2c1-eb94-4963-9a5b-03e9b6524ac2 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:46:14.697276 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-d703e2c1-eb94-4963-9a5b-03e9b6524ac2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7552545631e4947b34189a087b5457c'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:46:14.698977 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-d703e2c1-eb94-4963-9a5b-03e9b6524ac2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7552545631e4947b34189a087b5457c'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:46:14.714609 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-d703e2c1-eb94-4963-9a5b-03e9b6524ac2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7552545631e4947b34189a087b5457c'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:46:14.715448 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-d703e2c1-eb94-4963-9a5b-03e9b6524ac2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7552545631e4947b34189a087b5457c'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:46:14.826091 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-d703e2c1-eb94-4963-9a5b-03e9b6524ac2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7552545631e4947b34189a087b5457c'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:46:14.843847 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:46:14.852170 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 750/1507] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:46:14 2022] DELETE /identity/v3/users/f7552545631e4947b34189a087b5457c => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:46:14.857161 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ebcfa4-b666-48e5-806a-1339eaedc452 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:46:14.857649 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ebcfa4-b666-48e5-806a-1339eaedc452 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:46:14.857983 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ebcfa4-b666-48e5-806a-1339eaedc452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:46:14.876442 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-82ebcfa4-b666-48e5-806a-1339eaedc452 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:46:14.877671 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-82ebcfa4-b666-48e5-806a-1339eaedc452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:46:14.943175 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 758/1508] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:46:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:46:14.953705 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d703e2c1-eb94-4963-9a5b-03e9b6524ac2 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:46:14.954896 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-d703e2c1-eb94-4963-9a5b-03e9b6524ac2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:46:15.020486 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d703e2c1-eb94-4963-9a5b-03e9b6524ac2 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:46:15.024331 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6591683-9870-45c2-8195-bcd5622a6c49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:46:15.024989 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6591683-9870-45c2-8195-bcd5622a6c49 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:46:15.025278 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6591683-9870-45c2-8195-bcd5622a6c49 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:46:15.025557 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6591683-9870-45c2-8195-bcd5622a6c49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:46:15.035824 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6591683-9870-45c2-8195-bcd5622a6c49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:46:15.037347 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6591683-9870-45c2-8195-bcd5622a6c49 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:46:15.052080 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 751/1509] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:46:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:46:15.643848 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ebcfa4-b666-48e5-806a-1339eaedc452 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:46:15.648262 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-82ebcfa4-b666-48e5-806a-1339eaedc452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:46:15.664900 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ebcfa4-b666-48e5-806a-1339eaedc452 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:46:15.672819 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f79f20-c903-4949-8f9e-0d4cf0d39e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:46:15.673712 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f79f20-c903-4949-8f9e-0d4cf0d39e03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:46:15.673991 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f79f20-c903-4949-8f9e-0d4cf0d39e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:46:15.674267 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f79f20-c903-4949-8f9e-0d4cf0d39e03 admin admin] PATH_INFO: `/v3/projects/86a6a53d321f46a0bf8902180dc4ca46` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:46:15.675683 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f79f20-c903-4949-8f9e-0d4cf0d39e03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86a6a53d321f46a0bf8902180dc4ca46)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:46:15.676977 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: WARNING py.warnings [None req-04f79f20-c903-4949-8f9e-0d4cf0d39e03 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 14:46:15.676977 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: warnings.warn(msg) Jun 15 14:46:15.676977 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: Jun 15 14:46:15.678412 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f79f20-c903-4949-8f9e-0d4cf0d39e03 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:46:15.872945 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-04f79f20-c903-4949-8f9e-0d4cf0d39e03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86a6a53d321f46a0bf8902180dc4ca46', 'request_id': 'req-04f79f20-c903-4949-8f9e-0d4cf0d39e03'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:46:15.890177 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-04f79f20-c903-4949-8f9e-0d4cf0d39e03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86a6a53d321f46a0bf8902180dc4ca46', 'request_id': 'req-04f79f20-c903-4949-8f9e-0d4cf0d39e03'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:46:15.992018 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-04f79f20-c903-4949-8f9e-0d4cf0d39e03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86a6a53d321f46a0bf8902180dc4ca46', 'request_id': 'req-04f79f20-c903-4949-8f9e-0d4cf0d39e03'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:46:16.006013 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-04f79f20-c903-4949-8f9e-0d4cf0d39e03 admin admin] The token cache is being invalidate because project 86a6a53d321f46a0bf8902180dc4ca46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:46:16.006312 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-04f79f20-c903-4949-8f9e-0d4cf0d39e03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:46:16.007092 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-04f79f20-c903-4949-8f9e-0d4cf0d39e03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:46:16.008867 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 759/1510] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:46:15 2022] DELETE /identity/v3/projects/86a6a53d321f46a0bf8902180dc4ca46 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:46:25.043619 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:46:26.002517 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:48:00.117004 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb58f9-394d-4c76-9686-280551cc250a None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:00.117690 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb58f9-394d-4c76-9686-280551cc250a None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:00.117690 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb58f9-394d-4c76-9686-280551cc250a None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:00.119157 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 752/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 14:48:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:48:00.124786 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6939a53c-717a-451a-8b2a-b1baf322b6b3 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:00.125541 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6939a53c-717a-451a-8b2a-b1baf322b6b3 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:00.125541 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6939a53c-717a-451a-8b2a-b1baf322b6b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:00.146897 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-6939a53c-717a-451a-8b2a-b1baf322b6b3 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:48:00.148180 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-6939a53c-717a-451a-8b2a-b1baf322b6b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:48:00.221597 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 760/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 14:48:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:48:00.408344 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bb58f9-394d-4c76-9686-280551cc250a None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:48:00.416396 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-c8bb58f9-394d-4c76-9686-280551cc250a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:48:00.510622 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bb58f9-394d-4c76-9686-280551cc250a None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:48:00.516837 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660df346-00fd-42d4-bced-d633265ab3bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0927c05c340c48e48b18754888aa9c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:48:00.517558 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660df346-00fd-42d4-bced-d633265ab3bb service nova] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:00.517890 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660df346-00fd-42d4-bced-d633265ab3bb service nova] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:00.518216 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660df346-00fd-42d4-bced-d633265ab3bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:00.528371 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660df346-00fd-42d4-bced-d633265ab3bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:48:00.530104 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660df346-00fd-42d4-bced-d633265ab3bb service nova] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:48:00.542580 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 753/1513] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:48:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:48:01.481249 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9845cefc-2129-47f3-be79-7353ab2dcd0d None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:01.481870 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9845cefc-2129-47f3-be79-7353ab2dcd0d None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:01.481870 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9845cefc-2129-47f3-be79-7353ab2dcd0d None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:01.483194 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 761/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 14:48:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:48:01.489316 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f3b7b4-fd43-4942-9be2-040b94418a4b None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:01.489882 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f3b7b4-fd43-4942-9be2-040b94418a4b None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:01.490289 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f3b7b4-fd43-4942-9be2-040b94418a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:01.519105 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-d9f3b7b4-fd43-4942-9be2-040b94418a4b None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:48:01.520513 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-d9f3b7b4-fd43-4942-9be2-040b94418a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:48:01.528882 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 754/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 14:48:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:48:01.685524 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9845cefc-2129-47f3-be79-7353ab2dcd0d None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:48:01.705386 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9845cefc-2129-47f3-be79-7353ab2dcd0d None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:48:01.712630 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d27073-b164-47cc-a585-98101283efcd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0927c05c340c48e48b18754888aa9c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:48:01.713299 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d27073-b164-47cc-a585-98101283efcd service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:01.713584 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d27073-b164-47cc-a585-98101283efcd service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:01.713870 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d27073-b164-47cc-a585-98101283efcd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:01.723164 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d27073-b164-47cc-a585-98101283efcd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:48:01.724846 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d27073-b164-47cc-a585-98101283efcd service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:48:01.735589 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 762/1516] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:48:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:48:02.487294 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b958177-b442-42de-9c6d-257002bdee44 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:02.487580 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b958177-b442-42de-9c6d-257002bdee44 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:02.487860 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b958177-b442-42de-9c6d-257002bdee44 None None] PATH_INFO: `/v3` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:02.489316 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 755/1517] 10.222.0.46 () {54 vars in 925 bytes} [Wed Jun 15 14:48:02 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:48:02.496573 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b6f1dc-2d70-4dc3-b6c3-28bf6c6fa68b None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:02.497089 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b6f1dc-2d70-4dc3-b6c3-28bf6c6fa68b None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:02.497089 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b6f1dc-2d70-4dc3-b6c3-28bf6c6fa68b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:02.520456 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-89b6f1dc-2d70-4dc3-b6c3-28bf6c6fa68b None None] MFA Rules not processed for user `2bb6671c19464424a6e9cd4519ac93e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:48:02.521763 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-89b6f1dc-2d70-4dc3-b6c3-28bf6c6fa68b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:48:02.597110 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 763/1518] 10.222.0.46 () {58 vars in 1015 bytes} [Wed Jun 15 14:48:02 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:48:11.509836 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=37 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:48:12.439868 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920123ae-12fd-4ea9-856b-bdecbd9674f8 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:12.440080 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920123ae-12fd-4ea9-856b-bdecbd9674f8 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:12.440506 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920123ae-12fd-4ea9-856b-bdecbd9674f8 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:12.441748 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 756/1519] 10.222.0.42 () {54 vars in 960 bytes} [Wed Jun 15 14:48:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:48:12.447523 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e1ea48-4667-441b-8011-d279b0ca0ade None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:12.455237 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e1ea48-4667-441b-8011-d279b0ca0ade None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:12.455237 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e1ea48-4667-441b-8011-d279b0ca0ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:12.468710 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-99e1ea48-4667-441b-8011-d279b0ca0ade None None] MFA Rules not processed for user `2bb6671c19464424a6e9cd4519ac93e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:48:12.470857 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-99e1ea48-4667-441b-8011-d279b0ca0ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:48:12.479849 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 764/1520] 10.222.0.42 () {58 vars in 1057 bytes} [Wed Jun 15 14:48:12 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:48:12.497028 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920123ae-12fd-4ea9-856b-bdecbd9674f8 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:48:12.514528 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920123ae-12fd-4ea9-856b-bdecbd9674f8 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:48:12.520550 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a742e6-a8b6-4c1c-ba4e-6925b0c9d4e4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bb6671c19464424a6e9cd4519ac93e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:48:12.521211 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a742e6-a8b6-4c1c-ba4e-6925b0c9d4e4 service placement] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:12.521504 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a742e6-a8b6-4c1c-ba4e-6925b0c9d4e4 service placement] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:12.521814 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a742e6-a8b6-4c1c-ba4e-6925b0c9d4e4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:12.532743 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a742e6-a8b6-4c1c-ba4e-6925b0c9d4e4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:48:12.534377 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a742e6-a8b6-4c1c-ba4e-6925b0c9d4e4 service placement] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:48:12.550786 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 757/1521] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:48:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:48:13.060240 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72461733-5829-48cd-8a55-a59433acad56 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:13.060512 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72461733-5829-48cd-8a55-a59433acad56 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:13.060797 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72461733-5829-48cd-8a55-a59433acad56 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:13.062189 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 765/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 14:48:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:48:13.069742 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb887725-af3d-44e9-8867-b7c6b40fa937 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:13.070409 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb887725-af3d-44e9-8867-b7c6b40fa937 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:13.070692 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb887725-af3d-44e9-8867-b7c6b40fa937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:13.092121 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-cb887725-af3d-44e9-8867-b7c6b40fa937 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:48:13.093330 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-cb887725-af3d-44e9-8867-b7c6b40fa937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:48:13.102188 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 758/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 14:48:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:48:13.275527 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72461733-5829-48cd-8a55-a59433acad56 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:48:13.289561 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72461733-5829-48cd-8a55-a59433acad56 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:48:13.295236 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb7a4e2-5c84-4802-ad73-e5c889dde4a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0927c05c340c48e48b18754888aa9c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:48:13.295888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb7a4e2-5c84-4802-ad73-e5c889dde4a6 service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:13.296209 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb7a4e2-5c84-4802-ad73-e5c889dde4a6 service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:13.296491 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb7a4e2-5c84-4802-ad73-e5c889dde4a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:13.305940 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb7a4e2-5c84-4802-ad73-e5c889dde4a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:48:13.307559 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb7a4e2-5c84-4802-ad73-e5c889dde4a6 service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:48:13.317901 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 766/1524] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:48:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:48:14.209963 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de7d549-f036-4f8a-8a2a-7212db515a28 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:14.211117 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de7d549-f036-4f8a-8a2a-7212db515a28 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:14.211117 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de7d549-f036-4f8a-8a2a-7212db515a28 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:14.211971 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 759/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 14:48:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:48:14.217292 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b485cf-3f61-4473-8a39-0f89a177e0c8 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:14.217867 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b485cf-3f61-4473-8a39-0f89a177e0c8 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:14.217867 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b485cf-3f61-4473-8a39-0f89a177e0c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:14.242112 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-d5b485cf-3f61-4473-8a39-0f89a177e0c8 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:48:14.243334 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-d5b485cf-3f61-4473-8a39-0f89a177e0c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:48:14.252999 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 767/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 14:48:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:48:14.409078 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de7d549-f036-4f8a-8a2a-7212db515a28 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:48:14.425230 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de7d549-f036-4f8a-8a2a-7212db515a28 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:48:14.430680 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b744f2-b5ca-4d05-8bc2-1e15f411603d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0927c05c340c48e48b18754888aa9c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:48:14.431317 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b744f2-b5ca-4d05-8bc2-1e15f411603d service nova] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:14.431596 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b744f2-b5ca-4d05-8bc2-1e15f411603d service nova] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:14.431871 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b744f2-b5ca-4d05-8bc2-1e15f411603d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:14.440359 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b744f2-b5ca-4d05-8bc2-1e15f411603d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:48:14.441976 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b744f2-b5ca-4d05-8bc2-1e15f411603d service nova] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:48:14.452355 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 760/1527] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:48:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:48:18.510621 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698ceaa-e342-4eed-834d-1e93ee7abcfa None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:18.511214 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698ceaa-e342-4eed-834d-1e93ee7abcfa None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:18.511214 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698ceaa-e342-4eed-834d-1e93ee7abcfa None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:18.512878 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 768/1528] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 14:48:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:48:18.518896 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd8ed9c-6ddc-4461-9dd4-0118431468d7 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:18.519634 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd8ed9c-6ddc-4461-9dd4-0118431468d7 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:18.519634 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd8ed9c-6ddc-4461-9dd4-0118431468d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:18.538134 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-ebd8ed9c-6ddc-4461-9dd4-0118431468d7 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:48:18.539326 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-ebd8ed9c-6ddc-4461-9dd4-0118431468d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:48:18.547187 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 761/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 14:48:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:48:18.584295 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c698ceaa-e342-4eed-834d-1e93ee7abcfa None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:48:18.586169 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c698ceaa-e342-4eed-834d-1e93ee7abcfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:48:18.647610 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c698ceaa-e342-4eed-834d-1e93ee7abcfa None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:48:18.651477 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c292bd-2c1d-489a-a11f-0f3103e6bd57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:48:18.652147 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c292bd-2c1d-489a-a11f-0f3103e6bd57 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:18.652439 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c292bd-2c1d-489a-a11f-0f3103e6bd57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:18.652718 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c292bd-2c1d-489a-a11f-0f3103e6bd57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:18.662955 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c292bd-2c1d-489a-a11f-0f3103e6bd57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:48:18.664430 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c292bd-2c1d-489a-a11f-0f3103e6bd57 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:48:18.669603 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:48:18.675565 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 769/1530] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:48:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:48:20.877134 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43db6c46-7719-4a3b-affe-1af98c1647bf None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:20.877669 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43db6c46-7719-4a3b-affe-1af98c1647bf None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:20.877796 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43db6c46-7719-4a3b-affe-1af98c1647bf None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:20.879258 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 762/1531] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 14:48:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:48:20.884826 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c93b6-7dc4-4d08-8142-10c74b6c09ab None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:20.885264 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c93b6-7dc4-4d08-8142-10c74b6c09ab None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:20.885369 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c93b6-7dc4-4d08-8142-10c74b6c09ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:20.905256 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-0e6c93b6-7dc4-4d08-8142-10c74b6c09ab None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:48:20.906464 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-0e6c93b6-7dc4-4d08-8142-10c74b6c09ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:48:20.914767 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 770/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 14:48:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:48:20.954707 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43db6c46-7719-4a3b-affe-1af98c1647bf None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:48:20.956554 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-43db6c46-7719-4a3b-affe-1af98c1647bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:48:20.971282 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43db6c46-7719-4a3b-affe-1af98c1647bf None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:48:20.974848 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0ee6f0-0222-4785-9cb2-bc7f7a4dface service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:48:20.975483 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0ee6f0-0222-4785-9cb2-bc7f7a4dface service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:20.975768 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0ee6f0-0222-4785-9cb2-bc7f7a4dface service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:20.976035 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0ee6f0-0222-4785-9cb2-bc7f7a4dface service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:20.985890 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0ee6f0-0222-4785-9cb2-bc7f7a4dface service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:48:20.987325 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0ee6f0-0222-4785-9cb2-bc7f7a4dface service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:48:20.999741 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 763/1533] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:48:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:48:24.713215 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e67076-c941-4ea4-8cf7-f258e917d5d6 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:24.713805 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e67076-c941-4ea4-8cf7-f258e917d5d6 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:24.713805 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e67076-c941-4ea4-8cf7-f258e917d5d6 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:24.715162 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 771/1534] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 14:48:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:48:24.721363 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074922b4-bf42-4d09-9fb6-d6904243b0f2 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:24.722044 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074922b4-bf42-4d09-9fb6-d6904243b0f2 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:24.722044 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074922b4-bf42-4d09-9fb6-d6904243b0f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:24.741692 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-074922b4-bf42-4d09-9fb6-d6904243b0f2 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:48:24.742899 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-074922b4-bf42-4d09-9fb6-d6904243b0f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:48:24.750885 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 764/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 14:48:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:48:25.032530 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e67076-c941-4ea4-8cf7-f258e917d5d6 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:48:25.034284 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-08e67076-c941-4ea4-8cf7-f258e917d5d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:48:25.101931 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e67076-c941-4ea4-8cf7-f258e917d5d6 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:48:25.105592 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d078d-04fe-4c83-be1e-69ac570ee933 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:48:25.106364 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d078d-04fe-4c83-be1e-69ac570ee933 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:25.106647 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d078d-04fe-4c83-be1e-69ac570ee933 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:25.106923 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d078d-04fe-4c83-be1e-69ac570ee933 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:25.117225 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749d078d-04fe-4c83-be1e-69ac570ee933 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:48:25.118701 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749d078d-04fe-4c83-be1e-69ac570ee933 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:48:25.129636 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 772/1536] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:48:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:48:27.360242 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ca73ac-cbe3-4693-8867-21f5f25efccb None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:27.361101 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ca73ac-cbe3-4693-8867-21f5f25efccb None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:27.361101 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ca73ac-cbe3-4693-8867-21f5f25efccb None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:27.362276 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 765/1537] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 14:48:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:48:27.367467 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b563ba-4f60-4c94-8071-b01ece3cb954 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:27.367917 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b563ba-4f60-4c94-8071-b01ece3cb954 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:27.368024 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b563ba-4f60-4c94-8071-b01ece3cb954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:27.388927 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-a7b563ba-4f60-4c94-8071-b01ece3cb954 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:48:27.390219 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a7b563ba-4f60-4c94-8071-b01ece3cb954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:48:27.399419 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 773/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 14:48:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:48:27.671544 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ca73ac-cbe3-4693-8867-21f5f25efccb None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:48:27.674198 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-b7ca73ac-cbe3-4693-8867-21f5f25efccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:48:27.689127 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ca73ac-cbe3-4693-8867-21f5f25efccb None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:48:27.693019 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3036116e-98a2-4818-a7f4-edc1e00115d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:48:27.693711 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3036116e-98a2-4818-a7f4-edc1e00115d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:27.693999 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3036116e-98a2-4818-a7f4-edc1e00115d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:27.694297 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3036116e-98a2-4818-a7f4-edc1e00115d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:27.704921 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3036116e-98a2-4818-a7f4-edc1e00115d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:48:27.706398 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3036116e-98a2-4818-a7f4-edc1e00115d3 service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:48:27.721374 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 766/1539] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:48:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:48:37.383170 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=32 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:48:37.712081 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=33 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:00.363095 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251bf001-828d-47a7-bee7-baaa8c0b1123 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:00.363746 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251bf001-828d-47a7-bee7-baaa8c0b1123 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:00.363746 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251bf001-828d-47a7-bee7-baaa8c0b1123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:00.396101 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-251bf001-828d-47a7-bee7-baaa8c0b1123 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:00.397389 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-251bf001-828d-47a7-bee7-baaa8c0b1123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:00.414402 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 774/1540] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:50:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:00.431759 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d167766d-4e6a-4058-a8a9-11c74746a4d2 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:00.432452 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f317d70-97e6-409c-af46-83b28cde652e None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:00.432452 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f317d70-97e6-409c-af46-83b28cde652e None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:00.432452 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f317d70-97e6-409c-af46-83b28cde652e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:00.433286 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d167766d-4e6a-4058-a8a9-11c74746a4d2 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:00.433782 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d167766d-4e6a-4058-a8a9-11c74746a4d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:00.465315 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-4f317d70-97e6-409c-af46-83b28cde652e None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:00.466590 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-4f317d70-97e6-409c-af46-83b28cde652e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:00.473690 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-d167766d-4e6a-4058-a8a9-11c74746a4d2 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:00.475058 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-d167766d-4e6a-4058-a8a9-11c74746a4d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:00.481593 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 775/1541] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:50:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:00.493401 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9589cee-2456-43eb-b2fb-a9a44cd02e37 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:00.494164 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9589cee-2456-43eb-b2fb-a9a44cd02e37 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:00.494949 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9589cee-2456-43eb-b2fb-a9a44cd02e37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:00.502792 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 767/1542] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:50:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:50:00.504164 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d167766d-4e6a-4058-a8a9-11c74746a4d2 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:00.526048 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d167766d-4e6a-4058-a8a9-11c74746a4d2 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:00.533135 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-d9589cee-2456-43eb-b2fb-a9a44cd02e37 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:00.534828 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-d9589cee-2456-43eb-b2fb-a9a44cd02e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:00.542275 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c44e128-20e5-4bdf-9579-b57e29d2dc43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:00.542966 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c44e128-20e5-4bdf-9579-b57e29d2dc43 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:00.543251 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c44e128-20e5-4bdf-9579-b57e29d2dc43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:00.543530 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c44e128-20e5-4bdf-9579-b57e29d2dc43 admin admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:00.544295 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c44e128-20e5-4bdf-9579-b57e29d2dc43 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:50:00.544609 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c44e128-20e5-4bdf-9579-b57e29d2dc43 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:00.545653 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: WARNING py.warnings [None req-9c44e128-20e5-4bdf-9579-b57e29d2dc43 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 15 14:50:00.545653 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: warnings.warn(msg) Jun 15 14:50:00.545653 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: Jun 15 14:50:00.546738 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c44e128-20e5-4bdf-9579-b57e29d2dc43 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:00.571667 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 768/1543] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jun 15 14:50:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:50:00.573817 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 776/1544] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:50:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:00.575606 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048cfa3b-814a-4cde-a7f5-70a4e28a655d None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:00.575906 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048cfa3b-814a-4cde-a7f5-70a4e28a655d None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:00.576216 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048cfa3b-814a-4cde-a7f5-70a4e28a655d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:00.588429 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b49b88-d2df-4b5b-90cb-e6c853af3bd9 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:00.589069 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b49b88-d2df-4b5b-90cb-e6c853af3bd9 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:00.589572 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b49b88-d2df-4b5b-90cb-e6c853af3bd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:00.621307 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-048cfa3b-814a-4cde-a7f5-70a4e28a655d None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:00.622546 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-048cfa3b-814a-4cde-a7f5-70a4e28a655d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:00.632223 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-02b49b88-d2df-4b5b-90cb-e6c853af3bd9 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:00.633794 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-02b49b88-d2df-4b5b-90cb-e6c853af3bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:00.646674 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 769/1545] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:50:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:00.649033 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048cfa3b-814a-4cde-a7f5-70a4e28a655d None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:00.657353 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 777/1546] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:50:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:00.658548 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b49b88-d2df-4b5b-90cb-e6c853af3bd9 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:00.673284 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048cfa3b-814a-4cde-a7f5-70a4e28a655d None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:00.679755 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b49b88-d2df-4b5b-90cb-e6c853af3bd9 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:00.685685 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2758aea3-38d2-47fb-bca1-ad0088845360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:00.686569 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2758aea3-38d2-47fb-bca1-ad0088845360 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:00.686862 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2758aea3-38d2-47fb-bca1-ad0088845360 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:00.687148 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2758aea3-38d2-47fb-bca1-ad0088845360 admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:00.688033 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2758aea3-38d2-47fb-bca1-ad0088845360 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:00.690387 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2758aea3-38d2-47fb-bca1-ad0088845360 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:00.692676 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0eefff-e282-4584-9891-b5a2d1dc5c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:00.693788 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0eefff-e282-4584-9891-b5a2d1dc5c85 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:00.694739 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0eefff-e282-4584-9891-b5a2d1dc5c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:00.695282 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0eefff-e282-4584-9891-b5a2d1dc5c85 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:00.696329 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0eefff-e282-4584-9891-b5a2d1dc5c85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:50:00.696862 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0eefff-e282-4584-9891-b5a2d1dc5c85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:00.698760 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0eefff-e282-4584-9891-b5a2d1dc5c85 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:00.720768 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 778/1547] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jun 15 14:50:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:50:00.728133 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0eefff-e282-4584-9891-b5a2d1dc5c85 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:00.747387 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0eefff-e282-4584-9891-b5a2d1dc5c85 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:00.756799 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a73483-e7d1-4cf3-baf9-88d650ee6233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:00.759360 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a73483-e7d1-4cf3-baf9-88d650ee6233 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:00.759956 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a73483-e7d1-4cf3-baf9-88d650ee6233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:00.760538 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a73483-e7d1-4cf3-baf9-88d650ee6233 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:00.764157 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a73483-e7d1-4cf3-baf9-88d650ee6233 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:50:00.764157 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a73483-e7d1-4cf3-baf9-88d650ee6233 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:00.764157 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a73483-e7d1-4cf3-baf9-88d650ee6233 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:00.788232 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 779/1548] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jun 15 14:50:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:50:00.790042 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a73483-e7d1-4cf3-baf9-88d650ee6233 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:00.810572 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a73483-e7d1-4cf3-baf9-88d650ee6233 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:00.831476 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e972739e-3560-4797-a390-c13afe5bd5b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:00.832636 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e972739e-3560-4797-a390-c13afe5bd5b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:00.833131 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e972739e-3560-4797-a390-c13afe5bd5b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:00.833625 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e972739e-3560-4797-a390-c13afe5bd5b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:00.835000 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e972739e-3560-4797-a390-c13afe5bd5b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:00.837003 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: WARNING py.warnings [None req-e972739e-3560-4797-a390-c13afe5bd5b5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 14:50:00.837003 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: warnings.warn(msg) Jun 15 14:50:00.837003 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: Jun 15 14:50:00.839376 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e972739e-3560-4797-a390-c13afe5bd5b5 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:00.846687 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 770/1549] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:50:00 2022] POST /identity/v3/projects => generated 377 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:50:00.847453 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2758aea3-38d2-47fb-bca1-ad0088845360 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:00.977361 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 780/1550] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:50:00 2022] POST /identity/v3/projects => generated 371 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:50:00.984004 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e972739e-3560-4797-a390-c13afe5bd5b5 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:00.999249 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2758aea3-38d2-47fb-bca1-ad0088845360 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:01.083293 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e972739e-3560-4797-a390-c13afe5bd5b5 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:01.088212 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330a4bbc-3d74-4947-9641-43f454755ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:01.088893 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330a4bbc-3d74-4947-9641-43f454755ccb admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:01.089182 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330a4bbc-3d74-4947-9641-43f454755ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:01.089462 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330a4bbc-3d74-4947-9641-43f454755ccb admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:01.090175 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa445732-77a6-46cc-8fbd-9f2e0fc89703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:01.090318 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330a4bbc-3d74-4947-9641-43f454755ccb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:01.091309 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa445732-77a6-46cc-8fbd-9f2e0fc89703 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:01.091740 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa445732-77a6-46cc-8fbd-9f2e0fc89703 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:01.091934 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa445732-77a6-46cc-8fbd-9f2e0fc89703 admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:01.092156 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330a4bbc-3d74-4947-9641-43f454755ccb admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:01.096355 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa445732-77a6-46cc-8fbd-9f2e0fc89703 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:01.096355 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa445732-77a6-46cc-8fbd-9f2e0fc89703 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:01.208917 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 781/1551] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:50:00 2022] POST /identity/v3/users => generated 289 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:50:01.210885 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d289cd71-a1c8-4b18-a5e2-bf00e895dcb6 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:01.211310 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d289cd71-a1c8-4b18-a5e2-bf00e895dcb6 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:01.211594 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d289cd71-a1c8-4b18-a5e2-bf00e895dcb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:01.229558 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-d289cd71-a1c8-4b18-a5e2-bf00e895dcb6 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:01.235670 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-d289cd71-a1c8-4b18-a5e2-bf00e895dcb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:01.238896 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 771/1552] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:50:00 2022] POST /identity/v3/projects => generated 365 bytes in 392 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:50:01.240107 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa445732-77a6-46cc-8fbd-9f2e0fc89703 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:01.248176 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 782/1553] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:50:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:01.249904 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8d6b48-c364-4421-ab28-7c25da71ce9b None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:01.250814 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8d6b48-c364-4421-ab28-7c25da71ce9b None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:01.251113 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8d6b48-c364-4421-ab28-7c25da71ce9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:01.272077 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-5b8d6b48-c364-4421-ab28-7c25da71ce9b None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:01.274562 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-5b8d6b48-c364-4421-ab28-7c25da71ce9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:01.289252 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:01.305862 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa445732-77a6-46cc-8fbd-9f2e0fc89703 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:01.310689 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1198e56e-dabc-4cae-a179-d4ebdd6ecb0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:01.311365 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1198e56e-dabc-4cae-a179-d4ebdd6ecb0a admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:01.311638 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1198e56e-dabc-4cae-a179-d4ebdd6ecb0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:01.311920 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1198e56e-dabc-4cae-a179-d4ebdd6ecb0a admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:01.312652 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1198e56e-dabc-4cae-a179-d4ebdd6ecb0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:01.313688 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: WARNING py.warnings [None req-1198e56e-dabc-4cae-a179-d4ebdd6ecb0a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 14:50:01.313688 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: warnings.warn(msg) Jun 15 14:50:01.313688 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: Jun 15 14:50:01.315040 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1198e56e-dabc-4cae-a179-d4ebdd6ecb0a admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:01.338639 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 783/1554] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:50:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:01.339783 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32a244-eba7-49ed-a09d-7cd1034813b5 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:01.340139 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32a244-eba7-49ed-a09d-7cd1034813b5 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:01.340382 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32a244-eba7-49ed-a09d-7cd1034813b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:01.358924 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-5f32a244-eba7-49ed-a09d-7cd1034813b5 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:01.360165 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-5f32a244-eba7-49ed-a09d-7cd1034813b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:01.368995 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 784/1555] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:50:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:01.370300 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62741365-fc97-4aaa-9226-1138585d1442 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:01.370628 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62741365-fc97-4aaa-9226-1138585d1442 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:01.370960 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62741365-fc97-4aaa-9226-1138585d1442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:01.378436 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:01.391739 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-62741365-fc97-4aaa-9226-1138585d1442 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:01.392989 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-62741365-fc97-4aaa-9226-1138585d1442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:01.406040 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 785/1556] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:50:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:01.407580 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049a3920-5636-4ce0-8be6-c75883470369 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:01.407941 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049a3920-5636-4ce0-8be6-c75883470369 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:01.408265 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049a3920-5636-4ce0-8be6-c75883470369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:01.427555 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 772/1557] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:50:01 2022] POST /identity/v3/users => generated 286 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:50:01.427555 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1198e56e-dabc-4cae-a179-d4ebdd6ecb0a admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:01.430057 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-049a3920-5636-4ce0-8be6-c75883470369 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:01.431532 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-049a3920-5636-4ce0-8be6-c75883470369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:01.438168 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1198e56e-dabc-4cae-a179-d4ebdd6ecb0a admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:01.442517 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 786/1558] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:50:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:01.448143 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049a3920-5636-4ce0-8be6-c75883470369 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:01.448378 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85085ac0-81f1-4d8b-aac3-44af9c9d56fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:01.448378 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85085ac0-81f1-4d8b-aac3-44af9c9d56fc admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:01.448378 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85085ac0-81f1-4d8b-aac3-44af9c9d56fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:01.448378 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85085ac0-81f1-4d8b-aac3-44af9c9d56fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:01.448777 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85085ac0-81f1-4d8b-aac3-44af9c9d56fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:01.448777 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85085ac0-81f1-4d8b-aac3-44af9c9d56fc admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:01.463290 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049a3920-5636-4ce0-8be6-c75883470369 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:01.463510 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 773/1559] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:50:01 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:50:01.470835 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1893f84c-9a9c-45f7-be2e-b53096faf579 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:01.470835 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1893f84c-9a9c-45f7-be2e-b53096faf579 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:01.470835 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1893f84c-9a9c-45f7-be2e-b53096faf579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:01.471387 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9114fe6d-7903-453c-82b1-c395f353f80e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:01.471387 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9114fe6d-7903-453c-82b1-c395f353f80e admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:01.471387 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9114fe6d-7903-453c-82b1-c395f353f80e admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:01.471387 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9114fe6d-7903-453c-82b1-c395f353f80e admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:01.471998 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9114fe6d-7903-453c-82b1-c395f353f80e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:01.472694 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9114fe6d-7903-453c-82b1-c395f353f80e admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:01.487598 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-1893f84c-9a9c-45f7-be2e-b53096faf579 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:01.489399 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-1893f84c-9a9c-45f7-be2e-b53096faf579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:01.502696 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 774/1560] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:50:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:01.504120 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92ce251-94c1-4663-92fb-1d0866926627 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:01.504402 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92ce251-94c1-4663-92fb-1d0866926627 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:01.504676 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92ce251-94c1-4663-92fb-1d0866926627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:01.523973 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-b92ce251-94c1-4663-92fb-1d0866926627 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:01.525220 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-b92ce251-94c1-4663-92fb-1d0866926627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:01.533503 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 775/1561] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:50:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:01.534778 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c1dc74-e8e2-41a0-8bee-ea4e74aacd54 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:01.535065 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c1dc74-e8e2-41a0-8bee-ea4e74aacd54 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:01.535341 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c1dc74-e8e2-41a0-8bee-ea4e74aacd54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:01.555922 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-f0c1dc74-e8e2-41a0-8bee-ea4e74aacd54 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:01.557205 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-f0c1dc74-e8e2-41a0-8bee-ea4e74aacd54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:01.564282 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 787/1562] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:50:01 2022] POST /identity/v3/users => generated 283 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:50:01.564282 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c45a54e-82a8-43d8-bf8b-0950a8e7420d None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:01.564282 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c45a54e-82a8-43d8-bf8b-0950a8e7420d None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:01.564282 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c45a54e-82a8-43d8-bf8b-0950a8e7420d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:01.569287 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 776/1563] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:50:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:01.570926 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c1dc74-e8e2-41a0-8bee-ea4e74aacd54 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:01.580982 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c1dc74-e8e2-41a0-8bee-ea4e74aacd54 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:01.587295 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4774c2-a195-416c-bb65-c37f1c93ad51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:01.587760 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-4c45a54e-82a8-43d8-bf8b-0950a8e7420d None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:01.587977 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4774c2-a195-416c-bb65-c37f1c93ad51 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:01.588463 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4774c2-a195-416c-bb65-c37f1c93ad51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:01.589257 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4774c2-a195-416c-bb65-c37f1c93ad51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:01.589410 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-4c45a54e-82a8-43d8-bf8b-0950a8e7420d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:01.589623 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4774c2-a195-416c-bb65-c37f1c93ad51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:01.591039 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4774c2-a195-416c-bb65-c37f1c93ad51 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:01.597443 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 788/1564] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:50:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:01.598892 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8a1d6f-9e69-41b7-b541-2b8adc9a7ed8 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:01.599259 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8a1d6f-9e69-41b7-b541-2b8adc9a7ed8 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:01.599611 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8a1d6f-9e69-41b7-b541-2b8adc9a7ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:01.602917 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 777/1565] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:50:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:50:01.603640 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4774c2-a195-416c-bb65-c37f1c93ad51 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:01.616164 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4774c2-a195-416c-bb65-c37f1c93ad51 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:01.621604 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f6a4da-bdb7-4255-ac40-86fd9becd572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:01.622297 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f6a4da-bdb7-4255-ac40-86fd9becd572 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:01.622566 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f6a4da-bdb7-4255-ac40-86fd9becd572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:01.622849 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f6a4da-bdb7-4255-ac40-86fd9becd572 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:01.623579 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f6a4da-bdb7-4255-ac40-86fd9becd572 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:01.623950 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-7b8a1d6f-9e69-41b7-b541-2b8adc9a7ed8 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:01.625213 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-7b8a1d6f-9e69-41b7-b541-2b8adc9a7ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:01.625377 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f6a4da-bdb7-4255-ac40-86fd9becd572 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:01.633833 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 789/1566] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:50:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:01.634664 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8a1d6f-9e69-41b7-b541-2b8adc9a7ed8 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:01.636861 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 778/1567] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:50:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:50:01.638492 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f6a4da-bdb7-4255-ac40-86fd9becd572 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:01.649180 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8a1d6f-9e69-41b7-b541-2b8adc9a7ed8 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:01.650304 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f6a4da-bdb7-4255-ac40-86fd9becd572 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:01.654043 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d48726f-e5ea-4442-be55-47c34d22d6c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:01.654976 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d48726f-e5ea-4442-be55-47c34d22d6c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:01.655291 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d76459-5619-4aca-b5aa-868ff235a33e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:01.655465 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d48726f-e5ea-4442-be55-47c34d22d6c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:01.655592 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d48726f-e5ea-4442-be55-47c34d22d6c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:01.655953 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d76459-5619-4aca-b5aa-868ff235a33e admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:01.656297 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d48726f-e5ea-4442-be55-47c34d22d6c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:50:01.656415 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d76459-5619-4aca-b5aa-868ff235a33e admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:01.656590 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d76459-5619-4aca-b5aa-868ff235a33e admin admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:01.656689 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d48726f-e5ea-4442-be55-47c34d22d6c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:01.657332 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d76459-5619-4aca-b5aa-868ff235a33e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:50:01.657644 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d76459-5619-4aca-b5aa-868ff235a33e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:01.658217 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d48726f-e5ea-4442-be55-47c34d22d6c7 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:01.659272 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d76459-5619-4aca-b5aa-868ff235a33e admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:01.674005 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 779/1568] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jun 15 14:50:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:50:01.674211 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 790/1569] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jun 15 14:50:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:50:01.674757 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d48726f-e5ea-4442-be55-47c34d22d6c7 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:01.674885 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d76459-5619-4aca-b5aa-868ff235a33e admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:01.688114 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d48726f-e5ea-4442-be55-47c34d22d6c7 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:01.688358 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d76459-5619-4aca-b5aa-868ff235a33e admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:01.694005 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2fe7de-a12b-4f09-9e9b-493c1afc8307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:01.694231 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b36d8e4-82c8-4eb3-98aa-645d6ce78b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:01.695255 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b36d8e4-82c8-4eb3-98aa-645d6ce78b46 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:01.695387 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2fe7de-a12b-4f09-9e9b-493c1afc8307 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:01.695534 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b36d8e4-82c8-4eb3-98aa-645d6ce78b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:01.695854 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b36d8e4-82c8-4eb3-98aa-645d6ce78b46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:01.695988 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2fe7de-a12b-4f09-9e9b-493c1afc8307 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:01.696284 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2fe7de-a12b-4f09-9e9b-493c1afc8307 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:01.696561 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b36d8e4-82c8-4eb3-98aa-645d6ce78b46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:01.697025 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2fe7de-a12b-4f09-9e9b-493c1afc8307 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:50:01.697341 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2fe7de-a12b-4f09-9e9b-493c1afc8307 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:01.698157 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b36d8e4-82c8-4eb3-98aa-645d6ce78b46 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:01.698955 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2fe7de-a12b-4f09-9e9b-493c1afc8307 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:01.709598 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 780/1570] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:50:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:50:01.710733 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b36d8e4-82c8-4eb3-98aa-645d6ce78b46 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:01.715933 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 791/1571] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jun 15 14:50:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:50:01.716851 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2fe7de-a12b-4f09-9e9b-493c1afc8307 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:01.727581 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b36d8e4-82c8-4eb3-98aa-645d6ce78b46 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:01.727829 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2fe7de-a12b-4f09-9e9b-493c1afc8307 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:01.732521 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76af21e5-1f7e-4b38-bdb3-e33af9992f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:01.732683 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075d9700-3f5c-4244-9312-d87f7bb3ae3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:01.733219 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76af21e5-1f7e-4b38-bdb3-e33af9992f56 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:01.733353 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d9700-3f5c-4244-9312-d87f7bb3ae3b admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:01.733525 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76af21e5-1f7e-4b38-bdb3-e33af9992f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:01.733633 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d9700-3f5c-4244-9312-d87f7bb3ae3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:01.733803 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76af21e5-1f7e-4b38-bdb3-e33af9992f56 admin admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:01.733931 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d9700-3f5c-4244-9312-d87f7bb3ae3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:01.734720 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075d9700-3f5c-4244-9312-d87f7bb3ae3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:01.734884 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76af21e5-1f7e-4b38-bdb3-e33af9992f56 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:50:01.735294 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76af21e5-1f7e-4b38-bdb3-e33af9992f56 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:01.736363 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075d9700-3f5c-4244-9312-d87f7bb3ae3b admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:01.737126 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76af21e5-1f7e-4b38-bdb3-e33af9992f56 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:01.748385 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 792/1572] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:50:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:50:01.749279 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075d9700-3f5c-4244-9312-d87f7bb3ae3b admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:01.751802 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 781/1573] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jun 15 14:50:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:50:01.752670 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76af21e5-1f7e-4b38-bdb3-e33af9992f56 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:01.763843 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76af21e5-1f7e-4b38-bdb3-e33af9992f56 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:01.766504 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075d9700-3f5c-4244-9312-d87f7bb3ae3b admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:01.768811 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc33039d-dfa3-4245-a5ab-54dbceef909b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:01.769700 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc33039d-dfa3-4245-a5ab-54dbceef909b admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:01.769978 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc33039d-dfa3-4245-a5ab-54dbceef909b admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:01.770257 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc33039d-dfa3-4245-a5ab-54dbceef909b admin admin] PATH_INFO: `/v3/projects/0e498ac8b52d4a068016e44c094996c1/users/e95d993b6ed24073bbf1618ab4d1118e/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:01.771946 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f02925-80ea-4b28-bc3f-86dd53c72957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:01.772855 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f02925-80ea-4b28-bc3f-86dd53c72957 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:01.773214 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f02925-80ea-4b28-bc3f-86dd53c72957 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:01.773730 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f02925-80ea-4b28-bc3f-86dd53c72957 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:01.774740 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f02925-80ea-4b28-bc3f-86dd53c72957 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:50:01.775062 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f02925-80ea-4b28-bc3f-86dd53c72957 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:01.776767 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f02925-80ea-4b28-bc3f-86dd53c72957 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:01.789147 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc33039d-dfa3-4245-a5ab-54dbceef909b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e498ac8b52d4a068016e44c094996c1, user_id=e95d993b6ed24073bbf1618ab4d1118e, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:01.791617 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 793/1574] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jun 15 14:50:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:50:01.792243 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc33039d-dfa3-4245-a5ab-54dbceef909b admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:01.792497 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f02925-80ea-4b28-bc3f-86dd53c72957 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:01.809641 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f02925-80ea-4b28-bc3f-86dd53c72957 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:01.814405 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0ea910-e577-4262-8626-6d49274ec0dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:01.815056 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ea910-e577-4262-8626-6d49274ec0dd admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:01.815340 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ea910-e577-4262-8626-6d49274ec0dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:01.815611 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ea910-e577-4262-8626-6d49274ec0dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:01.816371 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0ea910-e577-4262-8626-6d49274ec0dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:01.818350 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0ea910-e577-4262-8626-6d49274ec0dd admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:01.877493 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 782/1575] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:50:01 2022] PUT /identity/v3/projects/0e498ac8b52d4a068016e44c094996c1/users/e95d993b6ed24073bbf1618ab4d1118e/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:01.878210 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc33039d-dfa3-4245-a5ab-54dbceef909b admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:01.939204 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 794/1576] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:50:01 2022] POST /identity/v3/projects => generated 375 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:50:01.940061 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0ea910-e577-4262-8626-6d49274ec0dd admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:01.943864 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc33039d-dfa3-4245-a5ab-54dbceef909b admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:01.976871 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:02.018456 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4608c41-372f-4d4a-8cd8-85b7e3d5f849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:02.019407 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4608c41-372f-4d4a-8cd8-85b7e3d5f849 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:02.019839 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4608c41-372f-4d4a-8cd8-85b7e3d5f849 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:02.020332 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4608c41-372f-4d4a-8cd8-85b7e3d5f849 admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:02.021046 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4608c41-372f-4d4a-8cd8-85b7e3d5f849 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:02.023199 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0ea910-e577-4262-8626-6d49274ec0dd admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:02.023980 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4608c41-372f-4d4a-8cd8-85b7e3d5f849 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:02.029535 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b82e0e-b4cb-4bd9-8dee-b4ad38bc6ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:02.030481 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b82e0e-b4cb-4bd9-8dee-b4ad38bc6ee4 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:02.030887 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b82e0e-b4cb-4bd9-8dee-b4ad38bc6ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:02.031259 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b82e0e-b4cb-4bd9-8dee-b4ad38bc6ee4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:02.032012 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b82e0e-b4cb-4bd9-8dee-b4ad38bc6ee4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:02.033694 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b82e0e-b4cb-4bd9-8dee-b4ad38bc6ee4 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:02.051119 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 795/1577] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:50:01 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:50:02.052429 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b82e0e-b4cb-4bd9-8dee-b4ad38bc6ee4 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:02.065695 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b82e0e-b4cb-4bd9-8dee-b4ad38bc6ee4 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:02.080068 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c3644-e1a1-44b1-ad41-ffb335b394a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:02.080789 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c3644-e1a1-44b1-ad41-ffb335b394a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:02.081072 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c3644-e1a1-44b1-ad41-ffb335b394a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:02.081356 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c3644-e1a1-44b1-ad41-ffb335b394a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:02.082170 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41c3644-e1a1-44b1-ad41-ffb335b394a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:02.084169 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41c3644-e1a1-44b1-ad41-ffb335b394a0 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:02.140987 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 783/1578] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:50:01 2022] POST /identity/v3/projects => generated 393 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:50:02.140987 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4608c41-372f-4d4a-8cd8-85b7e3d5f849 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:02.165601 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 796/1579] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:50:02 2022] POST /identity/v3/projects => generated 403 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:50:02.167700 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c3644-e1a1-44b1-ad41-ffb335b394a0 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:02.196602 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:02.222673 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4608c41-372f-4d4a-8cd8-85b7e3d5f849 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:02.258299 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c3644-e1a1-44b1-ad41-ffb335b394a0 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:02.266896 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95334f2-a815-4164-9ea0-3aef9af561ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:02.267711 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95334f2-a815-4164-9ea0-3aef9af561ae admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:02.268034 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95334f2-a815-4164-9ea0-3aef9af561ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:02.268358 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95334f2-a815-4164-9ea0-3aef9af561ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:02.269071 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95334f2-a815-4164-9ea0-3aef9af561ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:02.271013 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95334f2-a815-4164-9ea0-3aef9af561ae admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:02.297088 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718930e2-c1c5-451b-8383-0e5250abf45d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:02.297994 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718930e2-c1c5-451b-8383-0e5250abf45d admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:02.298278 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718930e2-c1c5-451b-8383-0e5250abf45d admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:02.298557 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718930e2-c1c5-451b-8383-0e5250abf45d admin admin] PATH_INFO: `/v3/projects/c30c40fc318d47069bea4c47036c6651/users/b1a53c440b9f4200b45a6b4f5f799305/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:02.316598 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718930e2-c1c5-451b-8383-0e5250abf45d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c30c40fc318d47069bea4c47036c6651, user_id=b1a53c440b9f4200b45a6b4f5f799305, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:02.319246 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718930e2-c1c5-451b-8383-0e5250abf45d admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:02.368315 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 797/1580] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:50:02 2022] POST /identity/v3/projects => generated 377 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:50:02.369155 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95334f2-a815-4164-9ea0-3aef9af561ae admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:02.421252 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 784/1581] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:50:02 2022] PUT /identity/v3/projects/c30c40fc318d47069bea4c47036c6651/users/b1a53c440b9f4200b45a6b4f5f799305/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:02.422648 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdcd9a6-f6d3-4f55-a4e8-596ae4bd20b6 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:02.423036 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdcd9a6-f6d3-4f55-a4e8-596ae4bd20b6 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:02.423574 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdcd9a6-f6d3-4f55-a4e8-596ae4bd20b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:02.446423 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95334f2-a815-4164-9ea0-3aef9af561ae admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:02.449169 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-1fdcd9a6-f6d3-4f55-a4e8-596ae4bd20b6 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:02.450390 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-1fdcd9a6-f6d3-4f55-a4e8-596ae4bd20b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:02.502458 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:02.530340 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 785/1582] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:50:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:02.530340 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdcd9a6-f6d3-4f55-a4e8-596ae4bd20b6 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:02.532679 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492b69b2-9d95-428c-bac8-888f0cb66c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:02.533798 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492b69b2-9d95-428c-bac8-888f0cb66c66 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:02.534322 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492b69b2-9d95-428c-bac8-888f0cb66c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:02.534830 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492b69b2-9d95-428c-bac8-888f0cb66c66 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:02.535582 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492b69b2-9d95-428c-bac8-888f0cb66c66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:02.537686 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492b69b2-9d95-428c-bac8-888f0cb66c66 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:02.544229 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdcd9a6-f6d3-4f55-a4e8-596ae4bd20b6 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:02.551420 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a15cf4-752d-44d6-a944-4c0a0815745b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:02.555598 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a15cf4-752d-44d6-a944-4c0a0815745b admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:02.555598 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a15cf4-752d-44d6-a944-4c0a0815745b admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:02.555598 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a15cf4-752d-44d6-a944-4c0a0815745b admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:02.555598 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a15cf4-752d-44d6-a944-4c0a0815745b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:02.556549 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a15cf4-752d-44d6-a944-4c0a0815745b admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:02.622581 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 798/1583] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:50:02 2022] POST /identity/v3/projects => generated 373 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:50:02.623411 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492b69b2-9d95-428c-bac8-888f0cb66c66 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:02.649680 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 786/1584] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:50:02 2022] POST /identity/v3/users => generated 288 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:50:02.650499 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a15cf4-752d-44d6-a944-4c0a0815745b admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:02.661324 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a15cf4-752d-44d6-a944-4c0a0815745b admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:02.679711 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:02.715832 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492b69b2-9d95-428c-bac8-888f0cb66c66 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:02.721600 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421c507c-4d9b-449f-9491-8bbf9917c87f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:02.723298 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c507c-4d9b-449f-9491-8bbf9917c87f admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:02.724004 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c507c-4d9b-449f-9491-8bbf9917c87f admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:02.724360 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c507c-4d9b-449f-9491-8bbf9917c87f admin admin] PATH_INFO: `/v3/projects/828c48ae467147b3a428f4273b63b0dc/users/0f85231ce9824ff8b2e690b5dbcb59e6/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:02.733894 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72665a30-bb9b-41fb-9b6a-ddfeaf37cc40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:02.736592 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72665a30-bb9b-41fb-9b6a-ddfeaf37cc40 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:02.736592 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72665a30-bb9b-41fb-9b6a-ddfeaf37cc40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:02.736592 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72665a30-bb9b-41fb-9b6a-ddfeaf37cc40 admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:02.736592 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72665a30-bb9b-41fb-9b6a-ddfeaf37cc40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:02.738079 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72665a30-bb9b-41fb-9b6a-ddfeaf37cc40 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:02.744344 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421c507c-4d9b-449f-9491-8bbf9917c87f admin admin] RBAC: Authorizing `identity:create_grant(project_id=828c48ae467147b3a428f4273b63b0dc, user_id=0f85231ce9824ff8b2e690b5dbcb59e6, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:02.745563 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: WARNING py.warnings [None req-421c507c-4d9b-449f-9491-8bbf9917c87f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 14:50:02.745563 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: warnings.warn(msg) Jun 15 14:50:02.745563 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: Jun 15 14:50:02.748483 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421c507c-4d9b-449f-9491-8bbf9917c87f admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:02.848757 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 799/1585] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:50:02 2022] PUT /identity/v3/projects/828c48ae467147b3a428f4273b63b0dc/users/0f85231ce9824ff8b2e690b5dbcb59e6/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:02.849571 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421c507c-4d9b-449f-9491-8bbf9917c87f admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:02.866050 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 787/1586] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:50:02 2022] POST /identity/v3/users => generated 297 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:50:02.867415 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72665a30-bb9b-41fb-9b6a-ddfeaf37cc40 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:02.877603 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72665a30-bb9b-41fb-9b6a-ddfeaf37cc40 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:02.916295 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421c507c-4d9b-449f-9491-8bbf9917c87f admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:02.921047 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404e8f8f-e237-401d-8d9e-0d6cf058b957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:02.921716 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404e8f8f-e237-401d-8d9e-0d6cf058b957 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:02.922000 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404e8f8f-e237-401d-8d9e-0d6cf058b957 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:02.922283 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404e8f8f-e237-401d-8d9e-0d6cf058b957 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:02.922970 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404e8f8f-e237-401d-8d9e-0d6cf058b957 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:02.924937 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404e8f8f-e237-401d-8d9e-0d6cf058b957 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:02.949227 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a027bd2b-cc65-4064-b9ee-9906747045e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:02.949227 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a027bd2b-cc65-4064-b9ee-9906747045e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:02.949568 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a027bd2b-cc65-4064-b9ee-9906747045e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:02.949802 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a027bd2b-cc65-4064-b9ee-9906747045e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:02.950548 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a027bd2b-cc65-4064-b9ee-9906747045e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:02.952591 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a027bd2b-cc65-4064-b9ee-9906747045e6 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:02.998022 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 800/1587] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:50:02 2022] POST /identity/v3/users => generated 302 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:50:02.999317 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bc9494-e760-4a1f-bb09-f2389fb29227 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:02.999611 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bc9494-e760-4a1f-bb09-f2389fb29227 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:02.999891 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bc9494-e760-4a1f-bb09-f2389fb29227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:03.032345 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-43bc9494-e760-4a1f-bb09-f2389fb29227 None None] MFA Rules not processed for user `b1a53c440b9f4200b45a6b4f5f799305`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:03.033589 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-43bc9494-e760-4a1f-bb09-f2389fb29227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:03.037771 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 788/1588] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:50:02 2022] POST /identity/v3/users => generated 289 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:50:03.037771 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a027bd2b-cc65-4064-b9ee-9906747045e6 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:03.086641 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:03.096403 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a027bd2b-cc65-4064-b9ee-9906747045e6 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:03.100025 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f2da34-5661-4ac3-8147-804879736909 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:03.101181 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 801/1589] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:50:02 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:03.101534 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f2da34-5661-4ac3-8147-804879736909 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:03.101823 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f2da34-5661-4ac3-8147-804879736909 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:03.102158 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f2da34-5661-4ac3-8147-804879736909 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:03.102266 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bc9494-e760-4a1f-bb09-f2389fb29227 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:03.112572 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bc9494-e760-4a1f-bb09-f2389fb29227 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:03.113407 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f2da34-5661-4ac3-8147-804879736909 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:03.118834 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f2da34-5661-4ac3-8147-804879736909 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:03.119011 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f54293e-5763-4c39-b797-1fb3f4d8db07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:03.120686 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f54293e-5763-4c39-b797-1fb3f4d8db07 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:03.120686 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f54293e-5763-4c39-b797-1fb3f4d8db07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:03.120686 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f54293e-5763-4c39-b797-1fb3f4d8db07 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:03.121031 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f54293e-5763-4c39-b797-1fb3f4d8db07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:03.123022 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f54293e-5763-4c39-b797-1fb3f4d8db07 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:03.129483 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 789/1590] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:50:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:50:03.130481 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f2da34-5661-4ac3-8147-804879736909 service neutron] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:03.141093 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f2da34-5661-4ac3-8147-804879736909 service neutron] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:03.146667 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259f0c8e-152c-457f-8c1b-e01aa4872416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:03.147347 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f0c8e-152c-457f-8c1b-e01aa4872416 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:03.148222 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f0c8e-152c-457f-8c1b-e01aa4872416 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:03.154338 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f0c8e-152c-457f-8c1b-e01aa4872416 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:03.154338 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259f0c8e-152c-457f-8c1b-e01aa4872416 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:03.154338 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259f0c8e-152c-457f-8c1b-e01aa4872416 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:03.162193 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 790/1591] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:50:03.164347 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416f81c5-38b0-4394-8143-e0f44540b824 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:03.164814 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416f81c5-38b0-4394-8143-e0f44540b824 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:03.165104 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416f81c5-38b0-4394-8143-e0f44540b824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:03.192461 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-416f81c5-38b0-4394-8143-e0f44540b824 None None] MFA Rules not processed for user `0f85231ce9824ff8b2e690b5dbcb59e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:03.194961 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-416f81c5-38b0-4394-8143-e0f44540b824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:03.248937 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 802/1592] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:50:03 2022] POST /identity/v3/users => generated 287 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:50:03.249802 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f54293e-5763-4c39-b797-1fb3f4d8db07 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:03.263728 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f54293e-5763-4c39-b797-1fb3f4d8db07 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:03.269104 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b12de0-139f-4016-b6f6-1de5d24b2c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:03.269829 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b12de0-139f-4016-b6f6-1de5d24b2c29 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:03.270113 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b12de0-139f-4016-b6f6-1de5d24b2c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:03.270389 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b12de0-139f-4016-b6f6-1de5d24b2c29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:03.271122 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b12de0-139f-4016-b6f6-1de5d24b2c29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:03.273454 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b12de0-139f-4016-b6f6-1de5d24b2c29 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:03.277613 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 791/1593] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:50:03 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:03.279013 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416f81c5-38b0-4394-8143-e0f44540b824 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:03.284456 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 803/1594] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:50:03.285443 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b12de0-139f-4016-b6f6-1de5d24b2c29 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:03.295695 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b12de0-139f-4016-b6f6-1de5d24b2c29 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:03.295941 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416f81c5-38b0-4394-8143-e0f44540b824 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:03.300877 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53baaa8-4170-44d5-8cb1-976891b3f2be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:03.301055 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd1f82f-9863-49d1-9666-1990a7ecca1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:03.301546 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53baaa8-4170-44d5-8cb1-976891b3f2be admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:03.301817 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd1f82f-9863-49d1-9666-1990a7ecca1b admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:03.301935 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53baaa8-4170-44d5-8cb1-976891b3f2be admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:03.302115 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd1f82f-9863-49d1-9666-1990a7ecca1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:03.302227 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53baaa8-4170-44d5-8cb1-976891b3f2be admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:03.302394 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd1f82f-9863-49d1-9666-1990a7ecca1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:03.302938 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53baaa8-4170-44d5-8cb1-976891b3f2be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:03.303121 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd1f82f-9863-49d1-9666-1990a7ecca1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:03.304702 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53baaa8-4170-44d5-8cb1-976891b3f2be admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:03.304862 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd1f82f-9863-49d1-9666-1990a7ecca1b admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:03.310339 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:03.317579 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 804/1595] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:50:03.318403 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53baaa8-4170-44d5-8cb1-976891b3f2be admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:03.324552 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 792/1596] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:50:03.324552 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd1f82f-9863-49d1-9666-1990a7ecca1b admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:03.329609 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53baaa8-4170-44d5-8cb1-976891b3f2be admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:03.334488 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7092e279-8765-4f89-90c2-791ac56d8053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:03.336118 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd1f82f-9863-49d1-9666-1990a7ecca1b admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:03.336251 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7092e279-8765-4f89-90c2-791ac56d8053 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:03.336465 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7092e279-8765-4f89-90c2-791ac56d8053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:03.336750 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7092e279-8765-4f89-90c2-791ac56d8053 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:03.337448 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7092e279-8765-4f89-90c2-791ac56d8053 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:03.339413 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7092e279-8765-4f89-90c2-791ac56d8053 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:03.341574 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e98c0c4-945b-4845-92fe-a7978541ce6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:03.342496 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e98c0c4-945b-4845-92fe-a7978541ce6e admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:03.342947 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e98c0c4-945b-4845-92fe-a7978541ce6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:03.343453 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e98c0c4-945b-4845-92fe-a7978541ce6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:03.344377 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e98c0c4-945b-4845-92fe-a7978541ce6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:03.346304 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e98c0c4-945b-4845-92fe-a7978541ce6e admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:03.359517 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 793/1597] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:50:03.360755 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e98c0c4-945b-4845-92fe-a7978541ce6e admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:03.371743 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e98c0c4-945b-4845-92fe-a7978541ce6e admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:03.376756 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c9fd15-839c-47c1-a057-b7cd65ffcd5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:03.377428 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c9fd15-839c-47c1-a057-b7cd65ffcd5b admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:03.377738 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c9fd15-839c-47c1-a057-b7cd65ffcd5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:03.378022 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c9fd15-839c-47c1-a057-b7cd65ffcd5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:03.378761 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c9fd15-839c-47c1-a057-b7cd65ffcd5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:03.380404 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c9fd15-839c-47c1-a057-b7cd65ffcd5b admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:03.394104 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 794/1598] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:50:03.394104 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c9fd15-839c-47c1-a057-b7cd65ffcd5b admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:03.406193 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c9fd15-839c-47c1-a057-b7cd65ffcd5b admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:03.413575 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4828b73-52f0-44b1-a1b5-e968b90dbf8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:03.414460 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4828b73-52f0-44b1-a1b5-e968b90dbf8c admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:03.414945 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4828b73-52f0-44b1-a1b5-e968b90dbf8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:03.415415 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4828b73-52f0-44b1-a1b5-e968b90dbf8c admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:03.416539 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4828b73-52f0-44b1-a1b5-e968b90dbf8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:03.421044 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4828b73-52f0-44b1-a1b5-e968b90dbf8c admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:03.461540 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 805/1599] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:50:03 2022] POST /identity/v3/projects => generated 373 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:50:03.461540 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7092e279-8765-4f89-90c2-791ac56d8053 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:03.539663 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 795/1600] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:50:03 2022] POST /identity/v3/projects => generated 369 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:50:03.539663 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4828b73-52f0-44b1-a1b5-e968b90dbf8c admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:03.540531 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7092e279-8765-4f89-90c2-791ac56d8053 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:03.614792 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4828b73-52f0-44b1-a1b5-e968b90dbf8c admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:03.616798 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e133ae0b-2d54-426e-9970-485f2b37f463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:03.617799 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e133ae0b-2d54-426e-9970-485f2b37f463 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:03.618243 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e133ae0b-2d54-426e-9970-485f2b37f463 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:03.618586 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e133ae0b-2d54-426e-9970-485f2b37f463 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:03.619714 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e133ae0b-2d54-426e-9970-485f2b37f463 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:03.620564 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8af8e5-3d06-4140-a9bd-d86fdb84f5b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:03.621231 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8af8e5-3d06-4140-a9bd-d86fdb84f5b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:03.621507 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8af8e5-3d06-4140-a9bd-d86fdb84f5b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:03.621777 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8af8e5-3d06-4140-a9bd-d86fdb84f5b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:03.621924 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e133ae0b-2d54-426e-9970-485f2b37f463 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:03.622499 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8af8e5-3d06-4140-a9bd-d86fdb84f5b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:03.624319 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8af8e5-3d06-4140-a9bd-d86fdb84f5b7 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:03.634660 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 806/1601] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:50:03.635823 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 796/1602] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:50:03.636819 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8af8e5-3d06-4140-a9bd-d86fdb84f5b7 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:03.641020 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e133ae0b-2d54-426e-9970-485f2b37f463 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:03.646755 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8af8e5-3d06-4140-a9bd-d86fdb84f5b7 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:03.647061 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e133ae0b-2d54-426e-9970-485f2b37f463 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:03.651801 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198e3f3d-7a0e-4938-83ff-d00280a4c823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:03.652369 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66808c7b-6aca-4a9f-95a9-bba5f2802656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:03.652797 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e3f3d-7a0e-4938-83ff-d00280a4c823 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:03.653074 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e3f3d-7a0e-4938-83ff-d00280a4c823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:03.653347 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e3f3d-7a0e-4938-83ff-d00280a4c823 admin admin] PATH_INFO: `/v3/projects/175b90b822a942bd8e04b8ad7d228a57/users/0650eb65117a4e27908c4355bfc46c3b/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:03.653450 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66808c7b-6aca-4a9f-95a9-bba5f2802656 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:03.653809 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66808c7b-6aca-4a9f-95a9-bba5f2802656 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:03.654186 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66808c7b-6aca-4a9f-95a9-bba5f2802656 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:03.655013 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66808c7b-6aca-4a9f-95a9-bba5f2802656 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:03.655626 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:03.657048 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66808c7b-6aca-4a9f-95a9-bba5f2802656 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:03.668385 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 807/1603] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:50:03.669306 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66808c7b-6aca-4a9f-95a9-bba5f2802656 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:03.669709 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198e3f3d-7a0e-4938-83ff-d00280a4c823 admin admin] RBAC: Authorizing `identity:create_grant(project_id=175b90b822a942bd8e04b8ad7d228a57, user_id=0650eb65117a4e27908c4355bfc46c3b, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:03.672371 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198e3f3d-7a0e-4938-83ff-d00280a4c823 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:03.680556 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66808c7b-6aca-4a9f-95a9-bba5f2802656 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:03.685462 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18918f7-5b82-45f2-8d0a-45e0f7f6a9c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:03.686174 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18918f7-5b82-45f2-8d0a-45e0f7f6a9c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:03.686516 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18918f7-5b82-45f2-8d0a-45e0f7f6a9c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:03.686846 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18918f7-5b82-45f2-8d0a-45e0f7f6a9c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:03.687593 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18918f7-5b82-45f2-8d0a-45e0f7f6a9c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:03.693073 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18918f7-5b82-45f2-8d0a-45e0f7f6a9c0 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:03.701463 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 808/1604] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:50:03.702291 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18918f7-5b82-45f2-8d0a-45e0f7f6a9c0 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:03.713331 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18918f7-5b82-45f2-8d0a-45e0f7f6a9c0 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:03.719100 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a800aaab-6813-4333-8bc7-85c4aa4f0080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:03.719796 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800aaab-6813-4333-8bc7-85c4aa4f0080 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:03.720498 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800aaab-6813-4333-8bc7-85c4aa4f0080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:03.725627 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800aaab-6813-4333-8bc7-85c4aa4f0080 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:03.725627 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a800aaab-6813-4333-8bc7-85c4aa4f0080 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:03.725627 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a800aaab-6813-4333-8bc7-85c4aa4f0080 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:03.753457 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 797/1605] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:50:03 2022] PUT /identity/v3/projects/175b90b822a942bd8e04b8ad7d228a57/users/0650eb65117a4e27908c4355bfc46c3b/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:03.754665 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198e3f3d-7a0e-4938-83ff-d00280a4c823 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:03.834785 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198e3f3d-7a0e-4938-83ff-d00280a4c823 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:03.837884 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689569b4-5760-45b9-9763-6117bcb2d342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:03.838567 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689569b4-5760-45b9-9763-6117bcb2d342 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:03.838846 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689569b4-5760-45b9-9763-6117bcb2d342 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:03.839121 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689569b4-5760-45b9-9763-6117bcb2d342 admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:03.839952 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689569b4-5760-45b9-9763-6117bcb2d342 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:03.842056 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689569b4-5760-45b9-9763-6117bcb2d342 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:03.863976 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 809/1606] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:50:03 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:50:03.863976 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a800aaab-6813-4333-8bc7-85c4aa4f0080 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:03.874491 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a800aaab-6813-4333-8bc7-85c4aa4f0080 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:03.879745 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17526645-e04f-4669-ba33-4d7aa2f4e145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:03.880831 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17526645-e04f-4669-ba33-4d7aa2f4e145 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:03.881149 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17526645-e04f-4669-ba33-4d7aa2f4e145 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:03.881486 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17526645-e04f-4669-ba33-4d7aa2f4e145 admin admin] PATH_INFO: `/v3/projects/b356eba3587e481aa04f99bcb2194b11/users/ee9df934d1cd461abbf9c3cd90a8fdda/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:03.899220 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17526645-e04f-4669-ba33-4d7aa2f4e145 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b356eba3587e481aa04f99bcb2194b11, user_id=ee9df934d1cd461abbf9c3cd90a8fdda, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:03.901928 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17526645-e04f-4669-ba33-4d7aa2f4e145 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:03.937761 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 798/1607] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:50:03 2022] POST /identity/v3/users => generated 291 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:50:03.937761 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689569b4-5760-45b9-9763-6117bcb2d342 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:03.947085 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689569b4-5760-45b9-9763-6117bcb2d342 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:03.952837 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46768c8b-2269-49be-8929-cd2951ecaca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:03.953262 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46768c8b-2269-49be-8929-cd2951ecaca7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:03.953770 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46768c8b-2269-49be-8929-cd2951ecaca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:03.954136 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46768c8b-2269-49be-8929-cd2951ecaca7 admin admin] PATH_INFO: `/v3/projects/11b1b9f71ff64d5ca24d195ca80eab2f/users/beb3dcb49d6f431298ce0056a972b50f/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:03.972381 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46768c8b-2269-49be-8929-cd2951ecaca7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11b1b9f71ff64d5ca24d195ca80eab2f, user_id=beb3dcb49d6f431298ce0056a972b50f, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:03.975427 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46768c8b-2269-49be-8929-cd2951ecaca7 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:04.015852 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 810/1608] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:50:03 2022] PUT /identity/v3/projects/b356eba3587e481aa04f99bcb2194b11/users/ee9df934d1cd461abbf9c3cd90a8fdda/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:04.016721 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17526645-e04f-4669-ba33-4d7aa2f4e145 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:04.042304 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:04.104337 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17526645-e04f-4669-ba33-4d7aa2f4e145 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:04.111143 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848259af-25ce-4ef5-8028-b6b7f0ac977d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:04.112065 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848259af-25ce-4ef5-8028-b6b7f0ac977d admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:04.112388 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848259af-25ce-4ef5-8028-b6b7f0ac977d admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:04.112675 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848259af-25ce-4ef5-8028-b6b7f0ac977d admin admin] PATH_INFO: `/v3/projects/0e0db8d8b4a54d9c9238f0332c61aa25/users/9034eb0a8e894a99a8316c60562c4f32/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:04.119979 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 799/1609] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:50:03 2022] PUT /identity/v3/projects/11b1b9f71ff64d5ca24d195ca80eab2f/users/beb3dcb49d6f431298ce0056a972b50f/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:04.120593 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46768c8b-2269-49be-8929-cd2951ecaca7 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:04.154411 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848259af-25ce-4ef5-8028-b6b7f0ac977d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e0db8d8b4a54d9c9238f0332c61aa25, user_id=9034eb0a8e894a99a8316c60562c4f32, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:04.160418 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848259af-25ce-4ef5-8028-b6b7f0ac977d admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:04.213636 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46768c8b-2269-49be-8929-cd2951ecaca7 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:04.218352 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4869e40d-b2fa-4fb6-be74-ef1143223eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:04.219233 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4869e40d-b2fa-4fb6-be74-ef1143223eb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:04.219512 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4869e40d-b2fa-4fb6-be74-ef1143223eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:04.219787 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4869e40d-b2fa-4fb6-be74-ef1143223eb3 admin admin] PATH_INFO: `/v3/projects/a0fa37ad549441d7b51b4463ab2fed53/users/24ffa9fc64d243dbae86bd4f1fdb7eff/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:04.237618 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4869e40d-b2fa-4fb6-be74-ef1143223eb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0fa37ad549441d7b51b4463ab2fed53, user_id=24ffa9fc64d243dbae86bd4f1fdb7eff, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:04.240340 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4869e40d-b2fa-4fb6-be74-ef1143223eb3 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:04.244900 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 811/1610] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:50:04 2022] PUT /identity/v3/projects/0e0db8d8b4a54d9c9238f0332c61aa25/users/9034eb0a8e894a99a8316c60562c4f32/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:04.244900 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cac592-a419-44d2-86d0-79e0ba929794 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:04.244900 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cac592-a419-44d2-86d0-79e0ba929794 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:04.244900 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cac592-a419-44d2-86d0-79e0ba929794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:04.260881 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-67cac592-a419-44d2-86d0-79e0ba929794 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:04.262346 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-67cac592-a419-44d2-86d0-79e0ba929794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:04.337341 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 812/1611] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:50:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:04.338294 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cac592-a419-44d2-86d0-79e0ba929794 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:04.356899 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cac592-a419-44d2-86d0-79e0ba929794 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:04.362848 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b3fba3-9c8f-44c6-873f-f3df58a644fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:04.363596 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b3fba3-9c8f-44c6-873f-f3df58a644fa admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:04.363884 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b3fba3-9c8f-44c6-873f-f3df58a644fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:04.364207 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b3fba3-9c8f-44c6-873f-f3df58a644fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:04.365015 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b3fba3-9c8f-44c6-873f-f3df58a644fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:04.366655 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b3fba3-9c8f-44c6-873f-f3df58a644fa admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:04.378824 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 813/1612] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:50:04 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:50:04.379685 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b3fba3-9c8f-44c6-873f-f3df58a644fa admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:04.390896 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b3fba3-9c8f-44c6-873f-f3df58a644fa admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:04.394922 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 800/1613] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:50:04 2022] PUT /identity/v3/projects/a0fa37ad549441d7b51b4463ab2fed53/users/24ffa9fc64d243dbae86bd4f1fdb7eff/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:04.405372 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bd17a6-00f3-4184-b300-411c3f999dd6 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:04.405917 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bd17a6-00f3-4184-b300-411c3f999dd6 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:04.406401 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bd17a6-00f3-4184-b300-411c3f999dd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:04.428754 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-72bd17a6-00f3-4184-b300-411c3f999dd6 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:04.432244 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-72bd17a6-00f3-4184-b300-411c3f999dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:04.461725 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:04.469996 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b50f6a-577d-42ad-a84a-67969426c4b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:04.470674 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b50f6a-577d-42ad-a84a-67969426c4b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:04.470958 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b50f6a-577d-42ad-a84a-67969426c4b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:04.471235 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b50f6a-577d-42ad-a84a-67969426c4b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:04.471961 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b50f6a-577d-42ad-a84a-67969426c4b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:04.473586 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b50f6a-577d-42ad-a84a-67969426c4b2 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:04.485476 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 814/1614] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:50:04 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:50:04.487181 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde34821-54ce-4113-85e8-12882cf37032 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:04.487482 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde34821-54ce-4113-85e8-12882cf37032 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:04.487768 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde34821-54ce-4113-85e8-12882cf37032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:04.499394 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 801/1615] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:50:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:04.500865 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d95bb1-a867-40d3-b8fb-ec0141dad12c None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:04.501812 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d95bb1-a867-40d3-b8fb-ec0141dad12c None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:04.501812 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d95bb1-a867-40d3-b8fb-ec0141dad12c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:04.524686 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-d0d95bb1-a867-40d3-b8fb-ec0141dad12c None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:04.525949 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-d0d95bb1-a867-40d3-b8fb-ec0141dad12c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:04.527769 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-dde34821-54ce-4113-85e8-12882cf37032 None None] MFA Rules not processed for user `beb3dcb49d6f431298ce0056a972b50f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:04.529307 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-dde34821-54ce-4113-85e8-12882cf37032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:04.535858 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 802/1616] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:50:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:04.536805 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d95bb1-a867-40d3-b8fb-ec0141dad12c None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:04.573307 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:04.652897 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d95bb1-a867-40d3-b8fb-ec0141dad12c None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:04.661257 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 815/1617] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:50:04 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:04.663048 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde34821-54ce-4113-85e8-12882cf37032 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:04.680650 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0b1f6c-21cd-4634-be25-4d9f9f041529 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:04.681803 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0b1f6c-21cd-4634-be25-4d9f9f041529 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:04.682303 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0b1f6c-21cd-4634-be25-4d9f9f041529 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:04.682795 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0b1f6c-21cd-4634-be25-4d9f9f041529 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:04.690589 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde34821-54ce-4113-85e8-12882cf37032 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:04.697636 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd92c9f-a9ae-4f0e-8bcb-f81d1b862826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:04.699173 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd92c9f-a9ae-4f0e-8bcb-f81d1b862826 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:04.699907 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd92c9f-a9ae-4f0e-8bcb-f81d1b862826 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:04.700447 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd92c9f-a9ae-4f0e-8bcb-f81d1b862826 admin admin] PATH_INFO: `/v3/projects/73c4418e87ee4a69a280297234cbe03b/users/c0b09650a7cd42f3a98ce4aa48ef89db/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:04.701249 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0b1f6c-21cd-4634-be25-4d9f9f041529 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:04.704393 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0b1f6c-21cd-4634-be25-4d9f9f041529 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:04.724290 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 803/1618] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:50:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:50:04.727531 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10288a7d-8a58-494b-9cce-fb92ade1b2c6 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:04.729422 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd92c9f-a9ae-4f0e-8bcb-f81d1b862826 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73c4418e87ee4a69a280297234cbe03b, user_id=c0b09650a7cd42f3a98ce4aa48ef89db, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:04.729602 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10288a7d-8a58-494b-9cce-fb92ade1b2c6 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:04.729883 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10288a7d-8a58-494b-9cce-fb92ade1b2c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:04.732631 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd92c9f-a9ae-4f0e-8bcb-f81d1b862826 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:04.765947 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-10288a7d-8a58-494b-9cce-fb92ade1b2c6 None None] MFA Rules not processed for user `24ffa9fc64d243dbae86bd4f1fdb7eff`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:04.767216 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-10288a7d-8a58-494b-9cce-fb92ade1b2c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:04.815788 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 816/1619] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:50:04 2022] PUT /identity/v3/projects/73c4418e87ee4a69a280297234cbe03b/users/c0b09650a7cd42f3a98ce4aa48ef89db/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:04.817510 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd92c9f-a9ae-4f0e-8bcb-f81d1b862826 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:04.855200 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 804/1620] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:50:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:04.855200 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10288a7d-8a58-494b-9cce-fb92ade1b2c6 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:04.863905 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10288a7d-8a58-494b-9cce-fb92ade1b2c6 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:04.923497 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd92c9f-a9ae-4f0e-8bcb-f81d1b862826 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:04.929127 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee068bb-2977-4c23-95b1-f3f8170a9b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:04.930329 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee068bb-2977-4c23-95b1-f3f8170a9b98 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:04.931270 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee068bb-2977-4c23-95b1-f3f8170a9b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:04.931270 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee068bb-2977-4c23-95b1-f3f8170a9b98 admin admin] PATH_INFO: `/v3/projects/31ddf4ec9b984d8d8475b84494a1e02a/users/df8679420d954bbf8c98504c6edb8866/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:04.933889 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d60601f-1d01-4df2-aa58-0e1685c2ab9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:04.934604 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d60601f-1d01-4df2-aa58-0e1685c2ab9b service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:04.934942 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d60601f-1d01-4df2-aa58-0e1685c2ab9b service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:04.935313 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d60601f-1d01-4df2-aa58-0e1685c2ab9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:04.948467 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d60601f-1d01-4df2-aa58-0e1685c2ab9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:04.950954 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d60601f-1d01-4df2-aa58-0e1685c2ab9b service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:04.954512 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee068bb-2977-4c23-95b1-f3f8170a9b98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31ddf4ec9b984d8d8475b84494a1e02a, user_id=df8679420d954bbf8c98504c6edb8866, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:04.957670 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:04.958348 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee068bb-2977-4c23-95b1-f3f8170a9b98 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:04.963494 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 805/1621] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:50:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:50:04.964261 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d60601f-1d01-4df2-aa58-0e1685c2ab9b service neutron] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:04.972154 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d60601f-1d01-4df2-aa58-0e1685c2ab9b service neutron] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:04.976359 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f4527a-59cb-44fc-9caf-eb6a324cffe2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:04.977048 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f4527a-59cb-44fc-9caf-eb6a324cffe2 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:04.977331 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f4527a-59cb-44fc-9caf-eb6a324cffe2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:04.977612 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f4527a-59cb-44fc-9caf-eb6a324cffe2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:04.989767 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f4527a-59cb-44fc-9caf-eb6a324cffe2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:04.992320 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f4527a-59cb-44fc-9caf-eb6a324cffe2 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:05.005104 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 806/1622] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:50:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:50:05.006024 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f4527a-59cb-44fc-9caf-eb6a324cffe2 service neutron] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:05.021102 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f4527a-59cb-44fc-9caf-eb6a324cffe2 service neutron] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:05.026737 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bccb1e-21a6-4415-a475-6fb1b9a1167e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:05.027381 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bccb1e-21a6-4415-a475-6fb1b9a1167e admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:05.027668 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bccb1e-21a6-4415-a475-6fb1b9a1167e admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:05.027948 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bccb1e-21a6-4415-a475-6fb1b9a1167e admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:05.028706 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bccb1e-21a6-4415-a475-6fb1b9a1167e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:05.030602 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bccb1e-21a6-4415-a475-6fb1b9a1167e admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:05.079339 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 817/1623] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:50:04 2022] PUT /identity/v3/projects/31ddf4ec9b984d8d8475b84494a1e02a/users/df8679420d954bbf8c98504c6edb8866/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:05.080747 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7533bf4-b921-42ae-aca9-19e28cc2a9c9 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:05.081083 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7533bf4-b921-42ae-aca9-19e28cc2a9c9 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:05.081481 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7533bf4-b921-42ae-aca9-19e28cc2a9c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:05.112605 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-e7533bf4-b921-42ae-aca9-19e28cc2a9c9 None None] MFA Rules not processed for user `e95d993b6ed24073bbf1618ab4d1118e`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:05.113847 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e7533bf4-b921-42ae-aca9-19e28cc2a9c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:05.168526 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 807/1624] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:50:05 2022] POST /identity/v3/projects => generated 377 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:50:05.168526 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bccb1e-21a6-4415-a475-6fb1b9a1167e admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:05.228997 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 818/1625] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:50:05 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:05.228997 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7533bf4-b921-42ae-aca9-19e28cc2a9c9 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:05.242313 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:05.272518 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bccb1e-21a6-4415-a475-6fb1b9a1167e admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:05.279742 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc306e42-e12d-4ebf-809d-321e754693e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:05.281427 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc306e42-e12d-4ebf-809d-321e754693e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:05.281993 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc306e42-e12d-4ebf-809d-321e754693e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:05.282373 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc306e42-e12d-4ebf-809d-321e754693e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:05.283221 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc306e42-e12d-4ebf-809d-321e754693e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:05.285005 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc306e42-e12d-4ebf-809d-321e754693e2 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:05.297744 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 808/1626] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:50:05 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:50:05.298852 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc306e42-e12d-4ebf-809d-321e754693e2 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:05.310268 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc306e42-e12d-4ebf-809d-321e754693e2 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:05.316386 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102b8a6f-5e47-4ba0-af0c-0a40ee819b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:05.317039 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102b8a6f-5e47-4ba0-af0c-0a40ee819b2b admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:05.317387 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102b8a6f-5e47-4ba0-af0c-0a40ee819b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:05.317757 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102b8a6f-5e47-4ba0-af0c-0a40ee819b2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:05.318485 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102b8a6f-5e47-4ba0-af0c-0a40ee819b2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:05.321227 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102b8a6f-5e47-4ba0-af0c-0a40ee819b2b admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:05.337583 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 809/1627] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:50:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:50:05.337776 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7533bf4-b921-42ae-aca9-19e28cc2a9c9 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:05.338582 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102b8a6f-5e47-4ba0-af0c-0a40ee819b2b admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:05.341221 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e51e2e9-4fd3-4066-a29d-c2cc7adb35c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:05.341836 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e51e2e9-4fd3-4066-a29d-c2cc7adb35c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:05.342457 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e51e2e9-4fd3-4066-a29d-c2cc7adb35c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:05.342457 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e51e2e9-4fd3-4066-a29d-c2cc7adb35c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:05.343089 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e51e2e9-4fd3-4066-a29d-c2cc7adb35c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:05.345062 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e51e2e9-4fd3-4066-a29d-c2cc7adb35c6 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:05.350195 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102b8a6f-5e47-4ba0-af0c-0a40ee819b2b admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:05.355261 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9b00aa-9c48-43b3-b0cc-5a5f9ab4482c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:05.355809 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b00aa-9c48-43b3-b0cc-5a5f9ab4482c admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:05.358615 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b00aa-9c48-43b3-b0cc-5a5f9ab4482c admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:05.358615 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b00aa-9c48-43b3-b0cc-5a5f9ab4482c admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:05.358615 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9b00aa-9c48-43b3-b0cc-5a5f9ab4482c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:05.359949 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9b00aa-9c48-43b3-b0cc-5a5f9ab4482c admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:05.459119 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 819/1628] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:50:05 2022] POST /identity/v3/projects => generated 373 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:50:05.461940 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d37dc5-94ad-475a-a4ac-c8f1e98d22dc None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:05.462592 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d37dc5-94ad-475a-a4ac-c8f1e98d22dc None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:05.463729 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d37dc5-94ad-475a-a4ac-c8f1e98d22dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:05.499806 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-56d37dc5-94ad-475a-a4ac-c8f1e98d22dc None None] MFA Rules not processed for user `e95d993b6ed24073bbf1618ab4d1118e`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:05.501153 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-56d37dc5-94ad-475a-a4ac-c8f1e98d22dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:05.515114 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 810/1629] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:50:05 2022] POST /identity/v3/users => generated 295 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:50:05.516526 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9b00aa-9c48-43b3-b0cc-5a5f9ab4482c admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:05.552128 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9b00aa-9c48-43b3-b0cc-5a5f9ab4482c admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:05.639138 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 820/1630] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:50:05 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:05.640199 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d37dc5-94ad-475a-a4ac-c8f1e98d22dc None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:05.657118 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced968f7-6aac-4aae-978b-5b392323ea4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:05.659817 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced968f7-6aac-4aae-978b-5b392323ea4d admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:05.660203 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced968f7-6aac-4aae-978b-5b392323ea4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:05.660506 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced968f7-6aac-4aae-978b-5b392323ea4d admin admin] PATH_INFO: `/v3/projects/73c4418e87ee4a69a280297234cbe03b/users/c0b09650a7cd42f3a98ce4aa48ef89db/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:05.662766 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced968f7-6aac-4aae-978b-5b392323ea4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=73c4418e87ee4a69a280297234cbe03b, user_id=c0b09650a7cd42f3a98ce4aa48ef89db, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:05.667969 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced968f7-6aac-4aae-978b-5b392323ea4d admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:05.686962 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 811/1631] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:50:05 2022] PUT /identity/v3/projects/73c4418e87ee4a69a280297234cbe03b/users/c0b09650a7cd42f3a98ce4aa48ef89db/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:05.697696 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced968f7-6aac-4aae-978b-5b392323ea4d admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:05.777080 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:05.779561 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d37dc5-94ad-475a-a4ac-c8f1e98d22dc None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:05.853387 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced968f7-6aac-4aae-978b-5b392323ea4d admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:05.870907 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834eed9c-16c8-46ed-8606-71d0ed4f3df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:05.871978 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834eed9c-16c8-46ed-8606-71d0ed4f3df5 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:05.873028 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834eed9c-16c8-46ed-8606-71d0ed4f3df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:05.873907 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834eed9c-16c8-46ed-8606-71d0ed4f3df5 admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:05.875085 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834eed9c-16c8-46ed-8606-71d0ed4f3df5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:05.877877 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:05.881426 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834eed9c-16c8-46ed-8606-71d0ed4f3df5 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:05.905188 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40746a2c-6848-4844-b1e5-5b5070290541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:05.906115 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40746a2c-6848-4844-b1e5-5b5070290541 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:05.906400 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40746a2c-6848-4844-b1e5-5b5070290541 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:05.906679 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40746a2c-6848-4844-b1e5-5b5070290541 admin admin] PATH_INFO: `/v3/projects/31ddf4ec9b984d8d8475b84494a1e02a/users/df8679420d954bbf8c98504c6edb8866/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:05.909133 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40746a2c-6848-4844-b1e5-5b5070290541 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31ddf4ec9b984d8d8475b84494a1e02a, user_id=df8679420d954bbf8c98504c6edb8866, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:05.911904 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40746a2c-6848-4844-b1e5-5b5070290541 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:05.936399 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 821/1632] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:50:05 2022] PUT /identity/v3/projects/31ddf4ec9b984d8d8475b84494a1e02a/users/df8679420d954bbf8c98504c6edb8866/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:05.936399 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40746a2c-6848-4844-b1e5-5b5070290541 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:06.007352 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 812/1633] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:50:05 2022] POST /identity/v3/users => generated 293 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:50:06.009558 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8969418b-9c2f-427f-b9d1-f6053443376a None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:06.010096 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8969418b-9c2f-427f-b9d1-f6053443376a None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:06.010711 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8969418b-9c2f-427f-b9d1-f6053443376a None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:06.021277 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 813/1634] 10.222.0.42 () {54 vars in 961 bytes} [Wed Jun 15 14:50:06 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:50:06.025241 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b8811-ecdb-4dc4-b1ab-6e9fc6be6fa9 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:06.025670 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b8811-ecdb-4dc4-b1ab-6e9fc6be6fa9 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:06.026090 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b8811-ecdb-4dc4-b1ab-6e9fc6be6fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:06.067674 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-591b8811-ecdb-4dc4-b1ab-6e9fc6be6fa9 None None] MFA Rules not processed for user `c0b09650a7cd42f3a98ce4aa48ef89db`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:06.069877 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-591b8811-ecdb-4dc4-b1ab-6e9fc6be6fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:06.151400 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40746a2c-6848-4844-b1e5-5b5070290541 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:06.174530 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498d22b3-b58f-4d24-a706-18fede51efff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:06.175616 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498d22b3-b58f-4d24-a706-18fede51efff admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:06.176183 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498d22b3-b58f-4d24-a706-18fede51efff admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:06.176634 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498d22b3-b58f-4d24-a706-18fede51efff admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:06.177982 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498d22b3-b58f-4d24-a706-18fede51efff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:06.182008 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498d22b3-b58f-4d24-a706-18fede51efff admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:06.187399 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 814/1635] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:50:06 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:06.190152 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2ae707-8349-4410-92a8-3c1f2ba97d0b None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:06.191111 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2ae707-8349-4410-92a8-3c1f2ba97d0b None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:06.192065 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2ae707-8349-4410-92a8-3c1f2ba97d0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:06.197938 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 822/1636] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:50:05 2022] GET /identity/v3/roles => generated 1420 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:50:06.199055 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498d22b3-b58f-4d24-a706-18fede51efff admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:06.226220 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498d22b3-b58f-4d24-a706-18fede51efff admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:06.243389 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bd02c5-320a-4409-8eb1-a42f51631a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:06.244604 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bd02c5-320a-4409-8eb1-a42f51631a90 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:06.245643 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bd02c5-320a-4409-8eb1-a42f51631a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:06.246201 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bd02c5-320a-4409-8eb1-a42f51631a90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:06.247208 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bd02c5-320a-4409-8eb1-a42f51631a90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:06.248202 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-3e2ae707-8349-4410-92a8-3c1f2ba97d0b None None] MFA Rules not processed for user `df8679420d954bbf8c98504c6edb8866`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:06.249479 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-3e2ae707-8349-4410-92a8-3c1f2ba97d0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:06.255174 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bd02c5-320a-4409-8eb1-a42f51631a90 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:06.283711 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 823/1637] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:50:06 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:50:06.289387 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7000eea-d80e-41c5-89d7-ebf235280df8 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:06.289387 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7000eea-d80e-41c5-89d7-ebf235280df8 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:06.289387 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7000eea-d80e-41c5-89d7-ebf235280df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:06.311428 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-d7000eea-d80e-41c5-89d7-ebf235280df8 None None] MFA Rules not processed for user `0927c05c340c48e48b18754888aa9c35`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:06.314437 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-d7000eea-d80e-41c5-89d7-ebf235280df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:06.353175 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 815/1638] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:50:06 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:06.354042 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2ae707-8349-4410-92a8-3c1f2ba97d0b None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:06.380580 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2ae707-8349-4410-92a8-3c1f2ba97d0b None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:06.387050 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91b77ba-40b9-4b1a-9638-072e629dc659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:06.390279 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91b77ba-40b9-4b1a-9638-072e629dc659 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:06.398198 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91b77ba-40b9-4b1a-9638-072e629dc659 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:06.398198 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91b77ba-40b9-4b1a-9638-072e629dc659 admin admin] PATH_INFO: `/v3/projects/e9fe1ff2fa2f4b32b74477c6e14cad5d/users/48433e0887a84e3ca10f507c3dbb6b29/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:06.431715 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91b77ba-40b9-4b1a-9638-072e629dc659 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9fe1ff2fa2f4b32b74477c6e14cad5d, user_id=48433e0887a84e3ca10f507c3dbb6b29, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:06.437863 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91b77ba-40b9-4b1a-9638-072e629dc659 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:06.443783 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 824/1639] 10.222.0.42 () {58 vars in 1058 bytes} [Wed Jun 15 14:50:06 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:06.449559 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7000eea-d80e-41c5-89d7-ebf235280df8 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:06.536838 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7000eea-d80e-41c5-89d7-ebf235280df8 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:06.542361 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095c7888-38cd-40ae-aac7-f0a60fcef414 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:06.543362 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c7888-38cd-40ae-aac7-f0a60fcef414 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:06.543845 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c7888-38cd-40ae-aac7-f0a60fcef414 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:06.544384 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c7888-38cd-40ae-aac7-f0a60fcef414 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:06.560651 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095c7888-38cd-40ae-aac7-f0a60fcef414 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:06.562155 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095c7888-38cd-40ae-aac7-f0a60fcef414 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:06.574254 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 825/1640] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:50:06 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:50:06.579703 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095c7888-38cd-40ae-aac7-f0a60fcef414 service cinder] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:06.587007 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095c7888-38cd-40ae-aac7-f0a60fcef414 service cinder] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:06.592724 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d955dfa-59a8-4f65-8e1c-606b1a095b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:06.593615 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d955dfa-59a8-4f65-8e1c-606b1a095b46 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:06.593900 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d955dfa-59a8-4f65-8e1c-606b1a095b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:06.594177 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d955dfa-59a8-4f65-8e1c-606b1a095b46 admin admin] PATH_INFO: `/v3/projects/3369fba032fd4fd88704530acfa673d8/users/e1a8c38ef80241fabae4572e11c1da62/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:06.615678 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 816/1641] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:50:06 2022] PUT /identity/v3/projects/e9fe1ff2fa2f4b32b74477c6e14cad5d/users/48433e0887a84e3ca10f507c3dbb6b29/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:06.615678 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91b77ba-40b9-4b1a-9638-072e629dc659 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:06.622080 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:06.626739 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d955dfa-59a8-4f65-8e1c-606b1a095b46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3369fba032fd4fd88704530acfa673d8, user_id=e1a8c38ef80241fabae4572e11c1da62, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:06.630347 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d955dfa-59a8-4f65-8e1c-606b1a095b46 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:06.687714 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91b77ba-40b9-4b1a-9638-072e629dc659 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:06.692444 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2ae560-c9dd-47f5-a0f3-908754539d73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:06.693497 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ae560-c9dd-47f5-a0f3-908754539d73 service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:06.693801 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ae560-c9dd-47f5-a0f3-908754539d73 service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:06.694085 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ae560-c9dd-47f5-a0f3-908754539d73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:06.770380 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 826/1642] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:50:06 2022] PUT /identity/v3/projects/3369fba032fd4fd88704530acfa673d8/users/e1a8c38ef80241fabae4572e11c1da62/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:06.775598 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d955dfa-59a8-4f65-8e1c-606b1a095b46 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:06.783638 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:06.786623 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2ae560-c9dd-47f5-a0f3-908754539d73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:06.797958 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2ae560-c9dd-47f5-a0f3-908754539d73 service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:06.828290 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 817/1643] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:50:06 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:50:06.829623 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2ae560-c9dd-47f5-a0f3-908754539d73 service cinder] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:06.912575 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d955dfa-59a8-4f65-8e1c-606b1a095b46 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:06.926086 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e5050d-9b30-4ecf-9cd7-e07cc417388a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0927c05c340c48e48b18754888aa9c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:06.927696 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e5050d-9b30-4ecf-9cd7-e07cc417388a service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:06.928455 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e5050d-9b30-4ecf-9cd7-e07cc417388a service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:06.930370 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e5050d-9b30-4ecf-9cd7-e07cc417388a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:06.958432 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2ae560-c9dd-47f5-a0f3-908754539d73 service cinder] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:07.073784 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e5050d-9b30-4ecf-9cd7-e07cc417388a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:07.077594 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e5050d-9b30-4ecf-9cd7-e07cc417388a service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:07.085924 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6388030-b234-4588-9711-bba5d5d783a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:07.087010 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6388030-b234-4588-9711-bba5d5d783a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:07.087579 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6388030-b234-4588-9711-bba5d5d783a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:07.088128 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6388030-b234-4588-9711-bba5d5d783a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:07.090213 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6388030-b234-4588-9711-bba5d5d783a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:07.093668 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6388030-b234-4588-9711-bba5d5d783a6 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:07.101735 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 827/1644] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:50:06 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:50:07.103452 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e5050d-9b30-4ecf-9cd7-e07cc417388a service nova] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:07.128657 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e5050d-9b30-4ecf-9cd7-e07cc417388a service nova] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:07.141901 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 818/1645] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:50:06 2022] GET /identity/v3/roles => generated 1420 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:50:07.141901 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6388030-b234-4588-9711-bba5d5d783a6 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:07.148854 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a254d2-c579-491d-af8c-0c1aa558a441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:07.152919 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a254d2-c579-491d-af8c-0c1aa558a441 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:07.152919 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a254d2-c579-491d-af8c-0c1aa558a441 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:07.152919 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a254d2-c579-491d-af8c-0c1aa558a441 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:07.152919 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a254d2-c579-491d-af8c-0c1aa558a441 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:07.155499 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a254d2-c579-491d-af8c-0c1aa558a441 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:07.187180 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 828/1646] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:50:07 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:50:07.187180 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a254d2-c579-491d-af8c-0c1aa558a441 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:07.195258 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a254d2-c579-491d-af8c-0c1aa558a441 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:07.203744 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174b6b5a-798b-424c-8ae2-28761001a32f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:07.204912 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174b6b5a-798b-424c-8ae2-28761001a32f admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:07.205409 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174b6b5a-798b-424c-8ae2-28761001a32f admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:07.206202 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174b6b5a-798b-424c-8ae2-28761001a32f admin admin] PATH_INFO: `/v3/projects/e9fe1ff2fa2f4b32b74477c6e14cad5d/users/48433e0887a84e3ca10f507c3dbb6b29/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:07.208931 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174b6b5a-798b-424c-8ae2-28761001a32f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9fe1ff2fa2f4b32b74477c6e14cad5d, user_id=48433e0887a84e3ca10f507c3dbb6b29, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:07.213107 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174b6b5a-798b-424c-8ae2-28761001a32f admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:07.240138 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 829/1647] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:50:07 2022] PUT /identity/v3/projects/e9fe1ff2fa2f4b32b74477c6e14cad5d/users/48433e0887a84e3ca10f507c3dbb6b29/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:07.240138 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174b6b5a-798b-424c-8ae2-28761001a32f admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:07.362591 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6388030-b234-4588-9711-bba5d5d783a6 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:07.420998 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174b6b5a-798b-424c-8ae2-28761001a32f admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:07.427515 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7096ec8-285a-4d4a-89f1-9ca93926ec5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:07.429670 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7096ec8-285a-4d4a-89f1-9ca93926ec5a admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:07.430549 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7096ec8-285a-4d4a-89f1-9ca93926ec5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:07.431098 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7096ec8-285a-4d4a-89f1-9ca93926ec5a admin admin] PATH_INFO: `/v3/projects/3369fba032fd4fd88704530acfa673d8/users/e1a8c38ef80241fabae4572e11c1da62/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:07.433874 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7096ec8-285a-4d4a-89f1-9ca93926ec5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3369fba032fd4fd88704530acfa673d8, user_id=e1a8c38ef80241fabae4572e11c1da62, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:07.437142 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7096ec8-285a-4d4a-89f1-9ca93926ec5a admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:07.453456 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 830/1648] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:50:07 2022] PUT /identity/v3/projects/3369fba032fd4fd88704530acfa673d8/users/e1a8c38ef80241fabae4572e11c1da62/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:07.453456 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e561f-c5de-499e-b841-9fb502be81cb None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:07.453456 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e561f-c5de-499e-b841-9fb502be81cb None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:07.453909 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e561f-c5de-499e-b841-9fb502be81cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:07.477574 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a89e128-d11d-41dc-8477-4ed8403f75cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:07.487764 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89e128-d11d-41dc-8477-4ed8403f75cb service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:07.487764 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89e128-d11d-41dc-8477-4ed8403f75cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:07.487764 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89e128-d11d-41dc-8477-4ed8403f75cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:07.565717 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-e83e561f-c5de-499e-b841-9fb502be81cb None None] MFA Rules not processed for user `48433e0887a84e3ca10f507c3dbb6b29`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:07.573905 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e83e561f-c5de-499e-b841-9fb502be81cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:07.613464 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a89e128-d11d-41dc-8477-4ed8403f75cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:07.618330 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a89e128-d11d-41dc-8477-4ed8403f75cb service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:07.654679 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 819/1649] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:50:07 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:50:07.680529 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f1bfc-91a1-4bac-8fd7-4b6686267f78 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:07.680885 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f1bfc-91a1-4bac-8fd7-4b6686267f78 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:07.681224 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f1bfc-91a1-4bac-8fd7-4b6686267f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:07.698231 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:07.712181 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 831/1650] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:50:07 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:07.712181 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad029a-73e2-4bf5-8334-88df45f9eee6 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:07.712181 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad029a-73e2-4bf5-8334-88df45f9eee6 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:07.712181 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad029a-73e2-4bf5-8334-88df45f9eee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:07.758292 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-ee8f1bfc-91a1-4bac-8fd7-4b6686267f78 None None] MFA Rules not processed for user `e1a8c38ef80241fabae4572e11c1da62`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:07.759554 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-ee8f1bfc-91a1-4bac-8fd7-4b6686267f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:07.767092 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:07.779414 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-bbad029a-73e2-4bf5-8334-88df45f9eee6 None None] MFA Rules not processed for user `9034eb0a8e894a99a8316c60562c4f32`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:07.784951 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-bbad029a-73e2-4bf5-8334-88df45f9eee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:07.881510 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 832/1651] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:50:07 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:07.881510 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cfa980-dc44-4f40-82a0-e378a102adaa None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:07.881510 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cfa980-dc44-4f40-82a0-e378a102adaa None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:07.881510 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cfa980-dc44-4f40-82a0-e378a102adaa None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:07.882945 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 833/1652] 10.222.0.42 () {54 vars in 912 bytes} [Wed Jun 15 14:50:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:50:07.883767 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cfa980-dc44-4f40-82a0-e378a102adaa None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:07.966812 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cfa980-dc44-4f40-82a0-e378a102adaa None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:07.981067 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 820/1653] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:50:07 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:07.981067 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8f1bfc-91a1-4bac-8fd7-4b6686267f78 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:07.986163 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f7147-23f2-4a9c-884c-d3193a999195 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:07.987755 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f7147-23f2-4a9c-884c-d3193a999195 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:07.988806 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f7147-23f2-4a9c-884c-d3193a999195 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:07.989291 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f7147-23f2-4a9c-884c-d3193a999195 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:08.068325 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:08.145979 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8f1bfc-91a1-4bac-8fd7-4b6686267f78 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:08.156069 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f108cc-7e10-4d55-a61e-761c8e39e36c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:08.157067 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f108cc-7e10-4d55-a61e-761c8e39e36c admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:08.159936 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f108cc-7e10-4d55-a61e-761c8e39e36c admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:08.159936 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f108cc-7e10-4d55-a61e-761c8e39e36c admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:08.160638 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f108cc-7e10-4d55-a61e-761c8e39e36c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:08.164775 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f108cc-7e10-4d55-a61e-761c8e39e36c admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:08.229142 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4f7147-23f2-4a9c-884c-d3193a999195 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:08.236732 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4f7147-23f2-4a9c-884c-d3193a999195 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:08.282102 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 834/1654] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:50:07 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:50:08.288504 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4230c231-de0b-430f-8900-e02faf935e38 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:08.288906 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4230c231-de0b-430f-8900-e02faf935e38 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:08.289100 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4230c231-de0b-430f-8900-e02faf935e38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:08.335762 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-4230c231-de0b-430f-8900-e02faf935e38 None None] MFA Rules not processed for user `0927c05c340c48e48b18754888aa9c35`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:08.339597 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-4230c231-de0b-430f-8900-e02faf935e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:08.355490 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 821/1655] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:50:07 2022] POST /identity/v3/projects => generated 403 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:50:08.360908 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f108cc-7e10-4d55-a61e-761c8e39e36c admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:08.528486 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:08.544802 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f108cc-7e10-4d55-a61e-761c8e39e36c admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:08.551282 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 835/1656] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Jun 15 14:50:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:08.565761 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4230c231-de0b-430f-8900-e02faf935e38 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:08.570718 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-4230c231-de0b-430f-8900-e02faf935e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:08.594147 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487d3b76-8e84-41a8-9f1d-e2178d006410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:08.595057 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d3b76-8e84-41a8-9f1d-e2178d006410 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:08.595398 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d3b76-8e84-41a8-9f1d-e2178d006410 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:08.595623 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d3b76-8e84-41a8-9f1d-e2178d006410 admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:08.597363 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487d3b76-8e84-41a8-9f1d-e2178d006410 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:08.599327 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487d3b76-8e84-41a8-9f1d-e2178d006410 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:08.715165 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4230c231-de0b-430f-8900-e02faf935e38 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:08.795605 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 822/1657] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:50:08 2022] POST /identity/v3/users => generated 308 bytes in 437 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:50:08.801840 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487d3b76-8e84-41a8-9f1d-e2178d006410 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:09.081705 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487d3b76-8e84-41a8-9f1d-e2178d006410 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:09.082900 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1991e32-cbbf-4670-a714-d5ab22b6cc26 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f18c57f27784694935bfb0bcea48a78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:09.082900 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1991e32-cbbf-4670-a714-d5ab22b6cc26 service glance] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:09.082900 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1991e32-cbbf-4670-a714-d5ab22b6cc26 service glance] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:09.082900 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1991e32-cbbf-4670-a714-d5ab22b6cc26 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:09.085389 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcc8ce9-c626-4601-9f04-d60fc9989b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:09.085389 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcc8ce9-c626-4601-9f04-d60fc9989b24 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:09.085389 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcc8ce9-c626-4601-9f04-d60fc9989b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:09.085389 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcc8ce9-c626-4601-9f04-d60fc9989b24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:09.086036 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcc8ce9-c626-4601-9f04-d60fc9989b24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:09.086036 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcc8ce9-c626-4601-9f04-d60fc9989b24 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:09.086036 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 823/1658] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:50:08 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:50:09.086036 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcc8ce9-c626-4601-9f04-d60fc9989b24 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:09.086036 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcc8ce9-c626-4601-9f04-d60fc9989b24 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:09.086583 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bd6de4-c5db-4bab-aa4f-255621f8bed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:09.086583 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bd6de4-c5db-4bab-aa4f-255621f8bed2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:09.086583 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bd6de4-c5db-4bab-aa4f-255621f8bed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:09.086583 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bd6de4-c5db-4bab-aa4f-255621f8bed2 admin admin] PATH_INFO: `/v3/projects/b44b498a2c0743969db45edc6828faf6/users/7a8af96d8573458f8070e35b63c52f23/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:09.087056 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bd6de4-c5db-4bab-aa4f-255621f8bed2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b44b498a2c0743969db45edc6828faf6, user_id=7a8af96d8573458f8070e35b63c52f23, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:09.087177 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1991e32-cbbf-4670-a714-d5ab22b6cc26 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:09.087313 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bd6de4-c5db-4bab-aa4f-255621f8bed2 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:09.087437 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1991e32-cbbf-4670-a714-d5ab22b6cc26 service glance] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:09.087437 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 836/1659] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:50:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:50:09.192865 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 824/1660] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:50:08 2022] PUT /identity/v3/projects/b44b498a2c0743969db45edc6828faf6/users/7a8af96d8573458f8070e35b63c52f23/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:50:09.199905 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1991e32-cbbf-4670-a714-d5ab22b6cc26 service glance] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:09.220617 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1991e32-cbbf-4670-a714-d5ab22b6cc26 service glance] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:09.317615 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc226483-74cc-4bc0-bf27-0abc6a724cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:09.318522 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc226483-74cc-4bc0-bf27-0abc6a724cdc admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:09.318894 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc226483-74cc-4bc0-bf27-0abc6a724cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:09.319949 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc226483-74cc-4bc0-bf27-0abc6a724cdc admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:09.320943 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc226483-74cc-4bc0-bf27-0abc6a724cdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:09.330807 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc226483-74cc-4bc0-bf27-0abc6a724cdc admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:09.350715 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 837/1661] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:50:09 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:50:09.357320 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bd6de4-c5db-4bab-aa4f-255621f8bed2 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:09.405807 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bd6de4-c5db-4bab-aa4f-255621f8bed2 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:09.414293 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fffed17-1302-4a7d-9678-a1cd5e8e8e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:09.416352 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fffed17-1302-4a7d-9678-a1cd5e8e8e76 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:09.416913 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fffed17-1302-4a7d-9678-a1cd5e8e8e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:09.417483 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fffed17-1302-4a7d-9678-a1cd5e8e8e76 admin admin] PATH_INFO: `/v3/projects/b44b498a2c0743969db45edc6828faf6/users/7a8af96d8573458f8070e35b63c52f23/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:09.439221 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fffed17-1302-4a7d-9678-a1cd5e8e8e76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b44b498a2c0743969db45edc6828faf6, user_id=7a8af96d8573458f8070e35b63c52f23, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:09.445925 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fffed17-1302-4a7d-9678-a1cd5e8e8e76 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:09.477368 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 825/1662] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:50:09 2022] PUT /identity/v3/projects/b44b498a2c0743969db45edc6828faf6/users/7a8af96d8573458f8070e35b63c52f23/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:09.532787 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc226483-74cc-4bc0-bf27-0abc6a724cdc admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:09.619279 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:09.712218 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc226483-74cc-4bc0-bf27-0abc6a724cdc admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:09.835664 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442c8dc3-078f-4030-a61d-86aafbc40ba2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:09.836896 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442c8dc3-078f-4030-a61d-86aafbc40ba2 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:09.837337 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442c8dc3-078f-4030-a61d-86aafbc40ba2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:09.837806 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442c8dc3-078f-4030-a61d-86aafbc40ba2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:09.985806 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442c8dc3-078f-4030-a61d-86aafbc40ba2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:09.988298 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442c8dc3-078f-4030-a61d-86aafbc40ba2 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:09.998249 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:10.006199 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 838/1663] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:50:09 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:50:10.263649 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b94e0a-8104-4b1c-a724-685747d78a14 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:10.264605 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b94e0a-8104-4b1c-a724-685747d78a14 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:10.265021 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b94e0a-8104-4b1c-a724-685747d78a14 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:10.267001 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 826/1664] 10.222.0.42 () {54 vars in 912 bytes} [Wed Jun 15 14:50:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:50:10.273275 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9725dff8-ee75-4b1b-b4ac-5cceb977ff46 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:10.273798 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9725dff8-ee75-4b1b-b4ac-5cceb977ff46 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:10.274278 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9725dff8-ee75-4b1b-b4ac-5cceb977ff46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:10.371106 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-9725dff8-ee75-4b1b-b4ac-5cceb977ff46 None None] MFA Rules not processed for user `c739b61c8a8b453e979bda720bafebb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:10.374005 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-9725dff8-ee75-4b1b-b4ac-5cceb977ff46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:10.499397 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 839/1665] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Jun 15 14:50:10 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:10.584538 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b94e0a-8104-4b1c-a724-685747d78a14 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:10.619243 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b94e0a-8104-4b1c-a724-685747d78a14 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:10.629126 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e4d599-f5a8-4dad-874b-07965fc89888 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:10.630705 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e4d599-f5a8-4dad-874b-07965fc89888 service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:10.630705 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e4d599-f5a8-4dad-874b-07965fc89888 service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:10.630705 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e4d599-f5a8-4dad-874b-07965fc89888 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:10.648404 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9725dff8-ee75-4b1b-b4ac-5cceb977ff46 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:10.662706 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9725dff8-ee75-4b1b-b4ac-5cceb977ff46 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:10.691448 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:10.691949 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73f0865-c3ae-488c-933c-a0457dcdfcd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:10.692710 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73f0865-c3ae-488c-933c-a0457dcdfcd8 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:10.692988 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73f0865-c3ae-488c-933c-a0457dcdfcd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:10.693262 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73f0865-c3ae-488c-933c-a0457dcdfcd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:10.734858 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73f0865-c3ae-488c-933c-a0457dcdfcd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:10.740819 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73f0865-c3ae-488c-933c-a0457dcdfcd8 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:10.768977 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 840/1666] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:50:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:50:10.830464 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e4d599-f5a8-4dad-874b-07965fc89888 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:10.832763 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e4d599-f5a8-4dad-874b-07965fc89888 service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:10.872621 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 827/1667] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:50:10 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:50:10.964037 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73f0865-c3ae-488c-933c-a0457dcdfcd8 service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:11.096288 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73f0865-c3ae-488c-933c-a0457dcdfcd8 service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:11.102953 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb93645-d2d4-4df5-89da-bf3f23a5ee4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:11.104066 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb93645-d2d4-4df5-89da-bf3f23a5ee4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:11.104597 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb93645-d2d4-4df5-89da-bf3f23a5ee4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:11.104883 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb93645-d2d4-4df5-89da-bf3f23a5ee4e admin admin] PATH_INFO: `/v3/users/b1a53c440b9f4200b45a6b4f5f799305` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:11.106234 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb93645-d2d4-4df5-89da-bf3f23a5ee4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1a53c440b9f4200b45a6b4f5f799305)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:11.107849 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: WARNING py.warnings [None req-3fb93645-d2d4-4df5-89da-bf3f23a5ee4e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 14:50:11.107849 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: warnings.warn(msg) Jun 15 14:50:11.107849 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: Jun 15 14:50:11.109613 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb93645-d2d4-4df5-89da-bf3f23a5ee4e admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:11.267971 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:11.472728 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-3fb93645-d2d4-4df5-89da-bf3f23a5ee4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1a53c440b9f4200b45a6b4f5f799305'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:11.473787 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-3fb93645-d2d4-4df5-89da-bf3f23a5ee4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1a53c440b9f4200b45a6b4f5f799305'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:11.479093 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-3fb93645-d2d4-4df5-89da-bf3f23a5ee4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1a53c440b9f4200b45a6b4f5f799305'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:11.654538 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-3fb93645-d2d4-4df5-89da-bf3f23a5ee4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1a53c440b9f4200b45a6b4f5f799305'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:11.702176 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-3fb93645-d2d4-4df5-89da-bf3f23a5ee4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1a53c440b9f4200b45a6b4f5f799305'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:11.768264 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 841/1668] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:50:10 2022] DELETE /identity/v3/users/b1a53c440b9f4200b45a6b4f5f799305 => generated 0 bytes in 811 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:11.771792 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19459d92-56b5-4c3f-bc8f-9d1ca9bb73c0 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:11.774170 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19459d92-56b5-4c3f-bc8f-9d1ca9bb73c0 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:11.775601 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19459d92-56b5-4c3f-bc8f-9d1ca9bb73c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:11.821726 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-19459d92-56b5-4c3f-bc8f-9d1ca9bb73c0 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:11.823509 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-19459d92-56b5-4c3f-bc8f-9d1ca9bb73c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:11.989236 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 828/1669] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:50:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:50:12.011218 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb93645-d2d4-4df5-89da-bf3f23a5ee4e admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:12.014445 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-3fb93645-d2d4-4df5-89da-bf3f23a5ee4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:12.239067 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb93645-d2d4-4df5-89da-bf3f23a5ee4e admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:12.251175 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a146dc2c-d9d9-422c-8ba9-697293fdd9c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:12.252017 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a146dc2c-d9d9-422c-8ba9-697293fdd9c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:12.253022 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a146dc2c-d9d9-422c-8ba9-697293fdd9c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:12.255224 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a146dc2c-d9d9-422c-8ba9-697293fdd9c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:12.281518 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a146dc2c-d9d9-422c-8ba9-697293fdd9c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:12.283649 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a146dc2c-d9d9-422c-8ba9-697293fdd9c1 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:12.299593 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 842/1670] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:50:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:50:12.320164 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faedaa43-fce3-4b0a-8fd7-fb00ffb5976a None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:12.326514 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faedaa43-fce3-4b0a-8fd7-fb00ffb5976a None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:12.330571 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faedaa43-fce3-4b0a-8fd7-fb00ffb5976a None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:12.337393 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 829/1671] 10.222.0.42 () {54 vars in 960 bytes} [Wed Jun 15 14:50:12 2022] GET /identity => generated 270 bytes in 24 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:50:12.344386 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9520de48-3ff5-4860-b415-70c24407130b None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:12.345438 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9520de48-3ff5-4860-b415-70c24407130b None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:12.346208 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9520de48-3ff5-4860-b415-70c24407130b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:12.448925 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-9520de48-3ff5-4860-b415-70c24407130b None None] MFA Rules not processed for user `2bb6671c19464424a6e9cd4519ac93e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:12.455691 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-9520de48-3ff5-4860-b415-70c24407130b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:12.743551 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 843/1672] 10.222.0.42 () {58 vars in 1057 bytes} [Wed Jun 15 14:50:12 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 401 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:12.744673 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faedaa43-fce3-4b0a-8fd7-fb00ffb5976a None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:12.785744 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faedaa43-fce3-4b0a-8fd7-fb00ffb5976a None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:12.801564 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d92d412-3bd2-4cf5-897d-b30c6fce15f2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bb6671c19464424a6e9cd4519ac93e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:12.802244 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d92d412-3bd2-4cf5-897d-b30c6fce15f2 service placement] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:12.802519 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d92d412-3bd2-4cf5-897d-b30c6fce15f2 service placement] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:12.802945 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d92d412-3bd2-4cf5-897d-b30c6fce15f2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:12.809953 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-4d92d412-3bd2-4cf5-897d-b30c6fce15f2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:12.839944 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d92d412-3bd2-4cf5-897d-b30c6fce15f2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:12.841820 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d92d412-3bd2-4cf5-897d-b30c6fce15f2 service placement] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:12.871769 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 830/1673] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:50:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:50:12.974764 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8534a431-093a-470f-8b25-794acee47dfc None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:12.974937 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8534a431-093a-470f-8b25-794acee47dfc None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:12.975172 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8534a431-093a-470f-8b25-794acee47dfc None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:12.977596 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 844/1674] 10.222.0.42 () {54 vars in 961 bytes} [Wed Jun 15 14:50:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:50:13.005079 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa369370-c89d-4bc4-a0ac-b02adba17290 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:13.005622 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa369370-c89d-4bc4-a0ac-b02adba17290 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:13.005897 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa369370-c89d-4bc4-a0ac-b02adba17290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:13.054233 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-fa369370-c89d-4bc4-a0ac-b02adba17290 None None] MFA Rules not processed for user `0927c05c340c48e48b18754888aa9c35`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:13.056217 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-fa369370-c89d-4bc4-a0ac-b02adba17290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:13.361057 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 831/1675] 10.222.0.42 () {58 vars in 1058 bytes} [Wed Jun 15 14:50:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:13.369023 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ab65a4-5a8f-4f33-9e5f-e2f58b4bb552 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:13.369320 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ab65a4-5a8f-4f33-9e5f-e2f58b4bb552 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:13.369596 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ab65a4-5a8f-4f33-9e5f-e2f58b4bb552 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:13.371243 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 845/1676] 10.222.0.42 () {54 vars in 912 bytes} [Wed Jun 15 14:50:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:50:13.377290 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d115af-9e9a-4480-9c39-c00902fa560a None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:13.377814 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d115af-9e9a-4480-9c39-c00902fa560a None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:13.378223 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d115af-9e9a-4480-9c39-c00902fa560a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:13.414222 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-f9d115af-9e9a-4480-9c39-c00902fa560a None None] MFA Rules not processed for user `0927c05c340c48e48b18754888aa9c35`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:13.415483 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-f9d115af-9e9a-4480-9c39-c00902fa560a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:13.432734 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 832/1677] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Jun 15 14:50:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:13.520323 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ab65a4-5a8f-4f33-9e5f-e2f58b4bb552 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:13.539629 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ab65a4-5a8f-4f33-9e5f-e2f58b4bb552 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:13.567226 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370e9be5-3564-4165-9a47-1d3fcddbbc71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:13.572833 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e9be5-3564-4165-9a47-1d3fcddbbc71 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:13.573426 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e9be5-3564-4165-9a47-1d3fcddbbc71 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:13.574060 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e9be5-3564-4165-9a47-1d3fcddbbc71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:13.626253 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370e9be5-3564-4165-9a47-1d3fcddbbc71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:13.629421 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370e9be5-3564-4165-9a47-1d3fcddbbc71 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:13.694658 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 846/1678] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:50:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:50:14.721159 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b033b1-8e2e-4874-90d8-87c8dca54390 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:14.721724 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b033b1-8e2e-4874-90d8-87c8dca54390 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:14.721724 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b033b1-8e2e-4874-90d8-87c8dca54390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:14.784280 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-38b033b1-8e2e-4874-90d8-87c8dca54390 None None] MFA Rules not processed for user `7a8af96d8573458f8070e35b63c52f23`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:14.786090 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-38b033b1-8e2e-4874-90d8-87c8dca54390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:14.834185 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:15.034384 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 833/1679] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:50:14 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:15.197642 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370e9be5-3564-4165-9a47-1d3fcddbbc71 service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:15.200512 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-370e9be5-3564-4165-9a47-1d3fcddbbc71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:15.245886 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370e9be5-3564-4165-9a47-1d3fcddbbc71 service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:15.267372 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb3f380-8f94-4105-8490-1ff5b85ecbe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:15.271401 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb3f380-8f94-4105-8490-1ff5b85ecbe4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:15.271913 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb3f380-8f94-4105-8490-1ff5b85ecbe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:15.272549 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb3f380-8f94-4105-8490-1ff5b85ecbe4 admin admin] PATH_INFO: `/v3/projects/c30c40fc318d47069bea4c47036c6651` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:15.293518 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb3f380-8f94-4105-8490-1ff5b85ecbe4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c30c40fc318d47069bea4c47036c6651)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:15.331743 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb3f380-8f94-4105-8490-1ff5b85ecbe4 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:15.476299 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:15.641570 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-2bb3f380-8f94-4105-8490-1ff5b85ecbe4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c30c40fc318d47069bea4c47036c6651', 'request_id': 'req-2bb3f380-8f94-4105-8490-1ff5b85ecbe4'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:15.679543 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-2bb3f380-8f94-4105-8490-1ff5b85ecbe4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c30c40fc318d47069bea4c47036c6651', 'request_id': 'req-2bb3f380-8f94-4105-8490-1ff5b85ecbe4'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:15.768777 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-2bb3f380-8f94-4105-8490-1ff5b85ecbe4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c30c40fc318d47069bea4c47036c6651', 'request_id': 'req-2bb3f380-8f94-4105-8490-1ff5b85ecbe4'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:15.787100 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-2bb3f380-8f94-4105-8490-1ff5b85ecbe4 admin admin] The token cache is being invalidate because project c30c40fc318d47069bea4c47036c6651 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:50:15.787401 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-2bb3f380-8f94-4105-8490-1ff5b85ecbe4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:15.788389 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-2bb3f380-8f94-4105-8490-1ff5b85ecbe4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:15.798969 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 847/1680] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:50:15 2022] DELETE /identity/v3/projects/c30c40fc318d47069bea4c47036c6651 => generated 0 bytes in 598 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:15.799869 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b033b1-8e2e-4874-90d8-87c8dca54390 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:15.864074 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b033b1-8e2e-4874-90d8-87c8dca54390 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:16.039840 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb3f380-8f94-4105-8490-1ff5b85ecbe4 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:16.042972 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-2bb3f380-8f94-4105-8490-1ff5b85ecbe4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:16.051615 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988df8d7-ac90-4c80-881b-372523fc9772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:16.052681 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988df8d7-ac90-4c80-881b-372523fc9772 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:16.053061 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988df8d7-ac90-4c80-881b-372523fc9772 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:16.053547 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988df8d7-ac90-4c80-881b-372523fc9772 admin admin] PATH_INFO: `/v3/users/c0b09650a7cd42f3a98ce4aa48ef89db` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:16.056076 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988df8d7-ac90-4c80-881b-372523fc9772 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0b09650a7cd42f3a98ce4aa48ef89db)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:16.061157 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988df8d7-ac90-4c80-881b-372523fc9772 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:16.077261 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb3f380-8f94-4105-8490-1ff5b85ecbe4 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:16.092991 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f239bf-2dc6-4769-bc1a-3de73cb97f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:16.107859 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f239bf-2dc6-4769-bc1a-3de73cb97f78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:16.111355 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f239bf-2dc6-4769-bc1a-3de73cb97f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:16.111816 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f239bf-2dc6-4769-bc1a-3de73cb97f78 admin admin] PATH_INFO: `/v3/users/0f85231ce9824ff8b2e690b5dbcb59e6` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:16.113656 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f239bf-2dc6-4769-bc1a-3de73cb97f78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f85231ce9824ff8b2e690b5dbcb59e6)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:16.124178 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f239bf-2dc6-4769-bc1a-3de73cb97f78 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:16.339523 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-988df8d7-ac90-4c80-881b-372523fc9772 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0b09650a7cd42f3a98ce4aa48ef89db'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:16.340375 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-988df8d7-ac90-4c80-881b-372523fc9772 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0b09650a7cd42f3a98ce4aa48ef89db'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:16.358935 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-988df8d7-ac90-4c80-881b-372523fc9772 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0b09650a7cd42f3a98ce4aa48ef89db'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:16.359823 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-988df8d7-ac90-4c80-881b-372523fc9772 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0b09650a7cd42f3a98ce4aa48ef89db'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:16.395055 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-95f239bf-2dc6-4769-bc1a-3de73cb97f78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f85231ce9824ff8b2e690b5dbcb59e6'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:16.398428 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-95f239bf-2dc6-4769-bc1a-3de73cb97f78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f85231ce9824ff8b2e690b5dbcb59e6'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:16.408670 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-95f239bf-2dc6-4769-bc1a-3de73cb97f78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f85231ce9824ff8b2e690b5dbcb59e6'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:16.443339 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-988df8d7-ac90-4c80-881b-372523fc9772 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0b09650a7cd42f3a98ce4aa48ef89db'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:16.499778 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 834/1681] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:50:15 2022] DELETE /identity/v3/users/c0b09650a7cd42f3a98ce4aa48ef89db => generated 0 bytes in 701 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:16.512358 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-95f239bf-2dc6-4769-bc1a-3de73cb97f78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f85231ce9824ff8b2e690b5dbcb59e6'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:16.538879 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-95f239bf-2dc6-4769-bc1a-3de73cb97f78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f85231ce9824ff8b2e690b5dbcb59e6'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:16.557675 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 848/1682] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:50:16 2022] DELETE /identity/v3/users/0f85231ce9824ff8b2e690b5dbcb59e6 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:16.562115 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391a9d7f-6821-4e92-a319-7f7030c3ae24 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:16.562366 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391a9d7f-6821-4e92-a319-7f7030c3ae24 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:16.562658 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391a9d7f-6821-4e92-a319-7f7030c3ae24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:16.587852 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-391a9d7f-6821-4e92-a319-7f7030c3ae24 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:16.590684 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-391a9d7f-6821-4e92-a319-7f7030c3ae24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:16.643327 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:16.731165 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 835/1683] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:50:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:16.772244 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f239bf-2dc6-4769-bc1a-3de73cb97f78 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:16.775248 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-95f239bf-2dc6-4769-bc1a-3de73cb97f78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:16.895941 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f239bf-2dc6-4769-bc1a-3de73cb97f78 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:16.901033 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e8575d-fa9d-4ec8-bc0e-f057b54f0956 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:16.901854 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e8575d-fa9d-4ec8-bc0e-f057b54f0956 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:16.902246 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e8575d-fa9d-4ec8-bc0e-f057b54f0956 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:16.902647 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e8575d-fa9d-4ec8-bc0e-f057b54f0956 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:16.923256 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e8575d-fa9d-4ec8-bc0e-f057b54f0956 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:16.925031 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e8575d-fa9d-4ec8-bc0e-f057b54f0956 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:16.945259 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 849/1684] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:50:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:50:16.993055 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dbe280-9b39-45ca-955d-907dfedcbd7c None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:16.994095 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dbe280-9b39-45ca-955d-907dfedcbd7c None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:16.994382 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dbe280-9b39-45ca-955d-907dfedcbd7c None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:16.998798 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 836/1685] 10.222.0.42 () {54 vars in 921 bytes} [Wed Jun 15 14:50:16 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:50:17.008840 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9226970-a36b-480a-85ca-917f1e845d77 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:17.009247 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9226970-a36b-480a-85ca-917f1e845d77 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:17.010128 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9226970-a36b-480a-85ca-917f1e845d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:17.046091 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-a9226970-a36b-480a-85ca-917f1e845d77 None None] MFA Rules not processed for user `2bb6671c19464424a6e9cd4519ac93e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:17.048235 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a9226970-a36b-480a-85ca-917f1e845d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:17.119174 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:17.197524 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 850/1686] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Jun 15 14:50:17 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:17.205201 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dbe280-9b39-45ca-955d-907dfedcbd7c None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:17.208239 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-d7dbe280-9b39-45ca-955d-907dfedcbd7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:17.232863 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dbe280-9b39-45ca-955d-907dfedcbd7c None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:17.239491 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8703b2-fc08-4735-a3bd-fc55b1b897cb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bb6671c19464424a6e9cd4519ac93e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:17.240584 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8703b2-fc08-4735-a3bd-fc55b1b897cb service placement] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:17.241542 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8703b2-fc08-4735-a3bd-fc55b1b897cb service placement] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:17.241960 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8703b2-fc08-4735-a3bd-fc55b1b897cb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:17.255449 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8703b2-fc08-4735-a3bd-fc55b1b897cb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:17.257937 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8703b2-fc08-4735-a3bd-fc55b1b897cb service placement] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:17.275450 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 837/1687] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:50:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:50:17.946866 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543f5e56-509f-4707-9975-7e9f397c86f0 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:17.948942 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543f5e56-509f-4707-9975-7e9f397c86f0 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:17.949249 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543f5e56-509f-4707-9975-7e9f397c86f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:18.058435 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-543f5e56-509f-4707-9975-7e9f397c86f0 None None] MFA Rules not processed for user `0650eb65117a4e27908c4355bfc46c3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:18.058435 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-543f5e56-509f-4707-9975-7e9f397c86f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:18.060499 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8703b2-fc08-4735-a3bd-fc55b1b897cb service placement] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:18.063207 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-4b8703b2-fc08-4735-a3bd-fc55b1b897cb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:18.140019 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8703b2-fc08-4735-a3bd-fc55b1b897cb service placement] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:18.152490 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eebb96-f038-4204-bbd5-463efd5516af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:18.153675 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eebb96-f038-4204-bbd5-463efd5516af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:18.163630 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eebb96-f038-4204-bbd5-463efd5516af admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:18.164205 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eebb96-f038-4204-bbd5-463efd5516af admin admin] PATH_INFO: `/v3/projects/73c4418e87ee4a69a280297234cbe03b` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:18.169614 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95eebb96-f038-4204-bbd5-463efd5516af admin admin] RBAC: Authorizing `identity:delete_project(project_id=73c4418e87ee4a69a280297234cbe03b)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:18.170862 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: WARNING py.warnings [None req-95eebb96-f038-4204-bbd5-463efd5516af admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 14:50:18.170862 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: warnings.warn(msg) Jun 15 14:50:18.170862 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: Jun 15 14:50:18.174705 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95eebb96-f038-4204-bbd5-463efd5516af admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:18.245920 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 851/1688] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:50:17 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:18.273201 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543f5e56-509f-4707-9975-7e9f397c86f0 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:18.274654 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-543f5e56-509f-4707-9975-7e9f397c86f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:18.448925 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-95eebb96-f038-4204-bbd5-463efd5516af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73c4418e87ee4a69a280297234cbe03b', 'request_id': 'req-95eebb96-f038-4204-bbd5-463efd5516af'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:18.488735 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543f5e56-509f-4707-9975-7e9f397c86f0 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:18.558262 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-95eebb96-f038-4204-bbd5-463efd5516af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73c4418e87ee4a69a280297234cbe03b', 'request_id': 'req-95eebb96-f038-4204-bbd5-463efd5516af'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:18.572021 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-95eebb96-f038-4204-bbd5-463efd5516af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73c4418e87ee4a69a280297234cbe03b', 'request_id': 'req-95eebb96-f038-4204-bbd5-463efd5516af'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:18.600643 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:18.625254 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-95eebb96-f038-4204-bbd5-463efd5516af admin admin] The token cache is being invalidate because project 73c4418e87ee4a69a280297234cbe03b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:50:18.625668 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-95eebb96-f038-4204-bbd5-463efd5516af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:18.628653 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-95eebb96-f038-4204-bbd5-463efd5516af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:18.638175 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 838/1689] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:50:18 2022] DELETE /identity/v3/projects/73c4418e87ee4a69a280297234cbe03b => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:18.640038 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eebb96-f038-4204-bbd5-463efd5516af admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:18.642430 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-95eebb96-f038-4204-bbd5-463efd5516af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:18.666384 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b8af3d-adba-4dd2-b44c-3ea5170aa781 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0927c05c340c48e48b18754888aa9c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:18.667066 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b8af3d-adba-4dd2-b44c-3ea5170aa781 service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:18.667540 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b8af3d-adba-4dd2-b44c-3ea5170aa781 service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:18.667810 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b8af3d-adba-4dd2-b44c-3ea5170aa781 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:18.672114 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-39b8af3d-adba-4dd2-b44c-3ea5170aa781 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:18.823040 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eebb96-f038-4204-bbd5-463efd5516af admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:18.832318 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b8af3d-adba-4dd2-b44c-3ea5170aa781 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:18.834058 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b8af3d-adba-4dd2-b44c-3ea5170aa781 service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:18.847224 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f877705-d385-4794-ae85-f9db9d82ab3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:18.848616 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f877705-d385-4794-ae85-f9db9d82ab3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:18.851917 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f877705-d385-4794-ae85-f9db9d82ab3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:18.852924 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f877705-d385-4794-ae85-f9db9d82ab3d admin admin] PATH_INFO: `/v3/projects/828c48ae467147b3a428f4273b63b0dc` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:18.861109 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f877705-d385-4794-ae85-f9db9d82ab3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=828c48ae467147b3a428f4273b63b0dc)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:18.863550 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f877705-d385-4794-ae85-f9db9d82ab3d admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:18.864276 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 852/1690] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:50:18 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 592 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:50:18.874471 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ea2389-d239-4b25-a7eb-756b49aea18d None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:18.874807 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ea2389-d239-4b25-a7eb-756b49aea18d None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:18.875095 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ea2389-d239-4b25-a7eb-756b49aea18d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:18.966161 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-e8ea2389-d239-4b25-a7eb-756b49aea18d None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:18.982021 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e8ea2389-d239-4b25-a7eb-756b49aea18d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:19.030862 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 853/1691] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:50:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:19.039251 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafbfeac-23c1-4e8c-9b35-794811e04f4d None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:19.040238 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafbfeac-23c1-4e8c-9b35-794811e04f4d None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:19.041564 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafbfeac-23c1-4e8c-9b35-794811e04f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:19.051625 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:19.063358 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-1f877705-d385-4794-ae85-f9db9d82ab3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '828c48ae467147b3a428f4273b63b0dc', 'request_id': 'req-1f877705-d385-4794-ae85-f9db9d82ab3d'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:19.113639 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-dafbfeac-23c1-4e8c-9b35-794811e04f4d None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:19.116199 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-dafbfeac-23c1-4e8c-9b35-794811e04f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:19.156890 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-1f877705-d385-4794-ae85-f9db9d82ab3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '828c48ae467147b3a428f4273b63b0dc', 'request_id': 'req-1f877705-d385-4794-ae85-f9db9d82ab3d'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:19.164826 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-1f877705-d385-4794-ae85-f9db9d82ab3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '828c48ae467147b3a428f4273b63b0dc', 'request_id': 'req-1f877705-d385-4794-ae85-f9db9d82ab3d'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:19.193817 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-1f877705-d385-4794-ae85-f9db9d82ab3d admin admin] The token cache is being invalidate because project 828c48ae467147b3a428f4273b63b0dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:50:19.195761 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-1f877705-d385-4794-ae85-f9db9d82ab3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:19.197090 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-1f877705-d385-4794-ae85-f9db9d82ab3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:19.205913 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 839/1692] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:50:18 2022] DELETE /identity/v3/projects/828c48ae467147b3a428f4273b63b0dc => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:19.211587 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f877705-d385-4794-ae85-f9db9d82ab3d admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:19.214468 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-1f877705-d385-4794-ae85-f9db9d82ab3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:19.311102 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 854/1693] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:50:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:19.311102 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc967e14-4046-4aa1-b0e7-2b02a0d52373 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:19.311102 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc967e14-4046-4aa1-b0e7-2b02a0d52373 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:19.311102 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc967e14-4046-4aa1-b0e7-2b02a0d52373 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:19.315850 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 855/1694] 10.222.0.42 () {54 vars in 921 bytes} [Wed Jun 15 14:50:19 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:50:19.316936 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc967e14-4046-4aa1-b0e7-2b02a0d52373 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:19.367216 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc967e14-4046-4aa1-b0e7-2b02a0d52373 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:19.374354 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59bb3b4-8d0d-41c7-b3c8-9bb6fecf7aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:19.375782 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59bb3b4-8d0d-41c7-b3c8-9bb6fecf7aeb admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:19.376217 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59bb3b4-8d0d-41c7-b3c8-9bb6fecf7aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:19.376803 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59bb3b4-8d0d-41c7-b3c8-9bb6fecf7aeb admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:19.378209 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59bb3b4-8d0d-41c7-b3c8-9bb6fecf7aeb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:50:19.379518 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59bb3b4-8d0d-41c7-b3c8-9bb6fecf7aeb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:19.384015 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59bb3b4-8d0d-41c7-b3c8-9bb6fecf7aeb admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:19.419250 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f877705-d385-4794-ae85-f9db9d82ab3d admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:19.435314 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf6f94b-575d-45d3-82d0-575ff358d3ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:19.435314 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf6f94b-575d-45d3-82d0-575ff358d3ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:19.435314 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf6f94b-575d-45d3-82d0-575ff358d3ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:19.435314 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf6f94b-575d-45d3-82d0-575ff358d3ba admin admin] PATH_INFO: `/v3/users/df8679420d954bbf8c98504c6edb8866` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:19.435662 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf6f94b-575d-45d3-82d0-575ff358d3ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=df8679420d954bbf8c98504c6edb8866)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:19.437598 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf6f94b-575d-45d3-82d0-575ff358d3ba admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:19.453234 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 856/1695] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jun 15 14:50:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:50:19.457276 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7129e1e3-7e83-4ac2-9a88-4f2fdbf0af90 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:19.457276 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7129e1e3-7e83-4ac2-9a88-4f2fdbf0af90 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:19.457621 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7129e1e3-7e83-4ac2-9a88-4f2fdbf0af90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:19.487056 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-7129e1e3-7e83-4ac2-9a88-4f2fdbf0af90 None None] MFA Rules not processed for user `2bb6671c19464424a6e9cd4519ac93e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:19.488681 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-7129e1e3-7e83-4ac2-9a88-4f2fdbf0af90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:19.645608 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 857/1696] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Jun 15 14:50:19 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:19.654286 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7129e1e3-7e83-4ac2-9a88-4f2fdbf0af90 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:19.659278 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-7129e1e3-7e83-4ac2-9a88-4f2fdbf0af90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:19.673958 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:19.684894 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7129e1e3-7e83-4ac2-9a88-4f2fdbf0af90 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:19.694994 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4917abd7-077d-4414-bc6e-dfb16ee989fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:19.697691 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917abd7-077d-4414-bc6e-dfb16ee989fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:19.698280 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917abd7-077d-4414-bc6e-dfb16ee989fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:19.698727 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917abd7-077d-4414-bc6e-dfb16ee989fb admin admin] PATH_INFO: `/v3/users/9034eb0a8e894a99a8316c60562c4f32` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:19.701131 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4917abd7-077d-4414-bc6e-dfb16ee989fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=9034eb0a8e894a99a8316c60562c4f32)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:19.705103 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4917abd7-077d-4414-bc6e-dfb16ee989fb admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:19.849470 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-eaf6f94b-575d-45d3-82d0-575ff358d3ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df8679420d954bbf8c98504c6edb8866'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:19.850465 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-eaf6f94b-575d-45d3-82d0-575ff358d3ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df8679420d954bbf8c98504c6edb8866'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:19.874012 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-eaf6f94b-575d-45d3-82d0-575ff358d3ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df8679420d954bbf8c98504c6edb8866'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:19.876611 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-eaf6f94b-575d-45d3-82d0-575ff358d3ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df8679420d954bbf8c98504c6edb8866'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:19.984536 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-eaf6f94b-575d-45d3-82d0-575ff358d3ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df8679420d954bbf8c98504c6edb8866'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:20.002901 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-4917abd7-077d-4414-bc6e-dfb16ee989fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9034eb0a8e894a99a8316c60562c4f32'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:20.003645 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-4917abd7-077d-4414-bc6e-dfb16ee989fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9034eb0a8e894a99a8316c60562c4f32'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:20.004949 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-4917abd7-077d-4414-bc6e-dfb16ee989fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9034eb0a8e894a99a8316c60562c4f32'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:20.007703 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 840/1697] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:50:19 2022] DELETE /identity/v3/users/df8679420d954bbf8c98504c6edb8866 => generated 0 bytes in 798 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:20.008984 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf6f94b-575d-45d3-82d0-575ff358d3ba admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:20.011798 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-eaf6f94b-575d-45d3-82d0-575ff358d3ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:20.077823 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-4917abd7-077d-4414-bc6e-dfb16ee989fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9034eb0a8e894a99a8316c60562c4f32'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:20.108719 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-4917abd7-077d-4414-bc6e-dfb16ee989fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9034eb0a8e894a99a8316c60562c4f32'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:20.110704 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:20.157720 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf6f94b-575d-45d3-82d0-575ff358d3ba admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:20.172989 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf4c1f1-bebf-4811-989c-6490331e67ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:20.173925 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf4c1f1-bebf-4811-989c-6490331e67ba admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:20.174241 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf4c1f1-bebf-4811-989c-6490331e67ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:20.174658 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf4c1f1-bebf-4811-989c-6490331e67ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:20.175673 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf4c1f1-bebf-4811-989c-6490331e67ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:20.182681 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf4c1f1-bebf-4811-989c-6490331e67ba admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:20.187024 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 858/1698] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:50:19 2022] DELETE /identity/v3/users/9034eb0a8e894a99a8316c60562c4f32 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:20.189589 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b35eb8b-7cff-4e69-b87a-78444b99ef09 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:20.191200 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b35eb8b-7cff-4e69-b87a-78444b99ef09 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:20.192990 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b35eb8b-7cff-4e69-b87a-78444b99ef09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:20.287884 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-9b35eb8b-7cff-4e69-b87a-78444b99ef09 None None] MFA Rules not processed for user `ee9df934d1cd461abbf9c3cd90a8fdda`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:20.290539 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-9b35eb8b-7cff-4e69-b87a-78444b99ef09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:20.381150 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 841/1699] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:50:20 2022] POST /identity/v3/projects => generated 355 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:50:20.388105 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf4c1f1-bebf-4811-989c-6490331e67ba admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:20.404481 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-dcf4c1f1-bebf-4811-989c-6490331e67ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:20.509377 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 859/1700] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:50:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:20.509645 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b35eb8b-7cff-4e69-b87a-78444b99ef09 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:20.513373 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-9b35eb8b-7cff-4e69-b87a-78444b99ef09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:20.700660 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf4c1f1-bebf-4811-989c-6490331e67ba admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:20.714996 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7153d312-7e5d-4bbf-8db5-e6711f3e7cd7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bb6671c19464424a6e9cd4519ac93e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:20.716523 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7153d312-7e5d-4bbf-8db5-e6711f3e7cd7 service placement] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:20.718173 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7153d312-7e5d-4bbf-8db5-e6711f3e7cd7 service placement] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:20.718901 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7153d312-7e5d-4bbf-8db5-e6711f3e7cd7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:20.723722 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-7153d312-7e5d-4bbf-8db5-e6711f3e7cd7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:20.760661 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7153d312-7e5d-4bbf-8db5-e6711f3e7cd7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:20.764788 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7153d312-7e5d-4bbf-8db5-e6711f3e7cd7 service placement] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:20.788742 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b35eb8b-7cff-4e69-b87a-78444b99ef09 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:20.800912 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 842/1701] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:50:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:50:20.803218 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bede3533-985c-4e3e-ae1c-f910d526d4ab None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:20.804409 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bede3533-985c-4e3e-ae1c-f910d526d4ab None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:20.804893 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bede3533-985c-4e3e-ae1c-f910d526d4ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:20.826958 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:20.888662 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-bede3533-985c-4e3e-ae1c-f910d526d4ab None None] MFA Rules not processed for user `ee9df934d1cd461abbf9c3cd90a8fdda`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:20.912539 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-bede3533-985c-4e3e-ae1c-f910d526d4ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:21.158812 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a43fee-88a2-4875-b9ef-e80afd0afc4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:21.158812 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a43fee-88a2-4875-b9ef-e80afd0afc4a admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:21.159863 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a43fee-88a2-4875-b9ef-e80afd0afc4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:21.159863 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a43fee-88a2-4875-b9ef-e80afd0afc4a admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:21.161097 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a43fee-88a2-4875-b9ef-e80afd0afc4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:21.170525 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a43fee-88a2-4875-b9ef-e80afd0afc4a admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:21.259315 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 843/1702] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:50:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 458 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:21.260909 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bede3533-985c-4e3e-ae1c-f910d526d4ab None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:21.266542 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-bede3533-985c-4e3e-ae1c-f910d526d4ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:21.303025 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bede3533-985c-4e3e-ae1c-f910d526d4ab None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:21.312257 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82abaeb-e7b8-4998-87cb-31c8bcde5101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:21.313791 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82abaeb-e7b8-4998-87cb-31c8bcde5101 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:21.314287 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82abaeb-e7b8-4998-87cb-31c8bcde5101 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:21.314772 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82abaeb-e7b8-4998-87cb-31c8bcde5101 admin admin] PATH_INFO: `/v3/projects/0e0db8d8b4a54d9c9238f0332c61aa25` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:21.317305 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82abaeb-e7b8-4998-87cb-31c8bcde5101 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e0db8d8b4a54d9c9238f0332c61aa25)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:21.329410 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82abaeb-e7b8-4998-87cb-31c8bcde5101 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:21.339328 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 860/1703] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:50:20 2022] POST /identity/v3/users => generated 278 bytes in 833 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:50:21.343793 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a43fee-88a2-4875-b9ef-e80afd0afc4a admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:21.356012 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-56a43fee-88a2-4875-b9ef-e80afd0afc4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:21.428607 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a43fee-88a2-4875-b9ef-e80afd0afc4a admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:21.447118 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bc7e16-6b04-4dbe-8aef-9bb856f3128d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:21.448953 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc7e16-6b04-4dbe-8aef-9bb856f3128d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:21.449482 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc7e16-6b04-4dbe-8aef-9bb856f3128d admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:21.451095 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc7e16-6b04-4dbe-8aef-9bb856f3128d admin admin] PATH_INFO: `/v3/projects/31ddf4ec9b984d8d8475b84494a1e02a` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:21.453732 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bc7e16-6b04-4dbe-8aef-9bb856f3128d admin admin] RBAC: Authorizing `identity:delete_project(project_id=31ddf4ec9b984d8d8475b84494a1e02a)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:21.459039 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bc7e16-6b04-4dbe-8aef-9bb856f3128d admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:21.611325 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-c82abaeb-e7b8-4998-87cb-31c8bcde5101 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e0db8d8b4a54d9c9238f0332c61aa25', 'request_id': 'req-c82abaeb-e7b8-4998-87cb-31c8bcde5101'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:21.696294 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-45bc7e16-6b04-4dbe-8aef-9bb856f3128d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31ddf4ec9b984d8d8475b84494a1e02a', 'request_id': 'req-45bc7e16-6b04-4dbe-8aef-9bb856f3128d'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:21.705204 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-45bc7e16-6b04-4dbe-8aef-9bb856f3128d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31ddf4ec9b984d8d8475b84494a1e02a', 'request_id': 'req-45bc7e16-6b04-4dbe-8aef-9bb856f3128d'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:21.710184 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-c82abaeb-e7b8-4998-87cb-31c8bcde5101 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e0db8d8b4a54d9c9238f0332c61aa25', 'request_id': 'req-c82abaeb-e7b8-4998-87cb-31c8bcde5101'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:21.734025 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-c82abaeb-e7b8-4998-87cb-31c8bcde5101 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e0db8d8b4a54d9c9238f0332c61aa25', 'request_id': 'req-c82abaeb-e7b8-4998-87cb-31c8bcde5101'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:21.734025 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:21.755380 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-c82abaeb-e7b8-4998-87cb-31c8bcde5101 admin admin] The token cache is being invalidate because project 0e0db8d8b4a54d9c9238f0332c61aa25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:50:21.755925 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-c82abaeb-e7b8-4998-87cb-31c8bcde5101 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:21.758825 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-c82abaeb-e7b8-4998-87cb-31c8bcde5101 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:21.765027 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 844/1704] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:50:21 2022] DELETE /identity/v3/projects/0e0db8d8b4a54d9c9238f0332c61aa25 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:21.765027 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82abaeb-e7b8-4998-87cb-31c8bcde5101 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:21.766929 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-c82abaeb-e7b8-4998-87cb-31c8bcde5101 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:21.838189 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-45bc7e16-6b04-4dbe-8aef-9bb856f3128d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31ddf4ec9b984d8d8475b84494a1e02a', 'request_id': 'req-45bc7e16-6b04-4dbe-8aef-9bb856f3128d'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:21.866287 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-45bc7e16-6b04-4dbe-8aef-9bb856f3128d admin admin] The token cache is being invalidate because project 31ddf4ec9b984d8d8475b84494a1e02a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:50:21.868752 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-45bc7e16-6b04-4dbe-8aef-9bb856f3128d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:21.871349 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-45bc7e16-6b04-4dbe-8aef-9bb856f3128d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:21.876835 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 861/1705] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:50:21 2022] DELETE /identity/v3/projects/31ddf4ec9b984d8d8475b84494a1e02a => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:21.884571 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bc7e16-6b04-4dbe-8aef-9bb856f3128d admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:21.888900 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-45bc7e16-6b04-4dbe-8aef-9bb856f3128d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:22.050225 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82abaeb-e7b8-4998-87cb-31c8bcde5101 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:22.155014 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bc7e16-6b04-4dbe-8aef-9bb856f3128d admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:22.163189 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194e653c-b1a4-42c2-b2f3-21d86fa6ff68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:22.165123 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e653c-b1a4-42c2-b2f3-21d86fa6ff68 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:22.165606 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e653c-b1a4-42c2-b2f3-21d86fa6ff68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:22.166363 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e653c-b1a4-42c2-b2f3-21d86fa6ff68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:22.169529 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194e653c-b1a4-42c2-b2f3-21d86fa6ff68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:22.178240 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194e653c-b1a4-42c2-b2f3-21d86fa6ff68 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:22.208216 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 862/1706] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:50:21 2022] GET /identity/v3/roles => generated 1420 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:50:22.211471 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194e653c-b1a4-42c2-b2f3-21d86fa6ff68 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:22.215612 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-194e653c-b1a4-42c2-b2f3-21d86fa6ff68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:22.251285 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:22.310666 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194e653c-b1a4-42c2-b2f3-21d86fa6ff68 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:22.315178 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceebaaf-eb9c-46bf-a214-c9974a0d3b48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:22.315943 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceebaaf-eb9c-46bf-a214-c9974a0d3b48 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:22.316347 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceebaaf-eb9c-46bf-a214-c9974a0d3b48 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:22.316638 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceebaaf-eb9c-46bf-a214-c9974a0d3b48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:22.317523 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1ee658-e0d4-40b4-a23b-0ab05789c22e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0927c05c340c48e48b18754888aa9c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:22.318681 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1ee658-e0d4-40b4-a23b-0ab05789c22e service nova] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:22.319748 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1ee658-e0d4-40b4-a23b-0ab05789c22e service nova] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:22.320885 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1ee658-e0d4-40b4-a23b-0ab05789c22e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:22.322332 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-0ceebaaf-eb9c-46bf-a214-c9974a0d3b48 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:22.324834 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-0c1ee658-e0d4-40b4-a23b-0ab05789c22e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:22.576177 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceebaaf-eb9c-46bf-a214-c9974a0d3b48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:22.579752 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceebaaf-eb9c-46bf-a214-c9974a0d3b48 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:22.619187 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 863/1707] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:50:22 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:50:22.632373 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceebaaf-eb9c-46bf-a214-c9974a0d3b48 service cinder] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:22.634122 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1ee658-e0d4-40b4-a23b-0ab05789c22e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:22.636179 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-0ceebaaf-eb9c-46bf-a214-c9974a0d3b48 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:22.638545 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1ee658-e0d4-40b4-a23b-0ab05789c22e service nova] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:22.675290 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 845/1708] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:50:21 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 912 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:50:22.678148 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83b094c-0d5e-4a41-ab29-a2815a95ea0c None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:22.679097 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83b094c-0d5e-4a41-ab29-a2815a95ea0c None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:22.679454 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83b094c-0d5e-4a41-ab29-a2815a95ea0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:22.690001 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceebaaf-eb9c-46bf-a214-c9974a0d3b48 service cinder] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:22.711976 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1608e39a-b92b-42d7-913f-491ee26d068c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:22.714292 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1608e39a-b92b-42d7-913f-491ee26d068c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:22.714583 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1608e39a-b92b-42d7-913f-491ee26d068c admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:22.714868 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1608e39a-b92b-42d7-913f-491ee26d068c admin admin] PATH_INFO: `/v3/users/7a8af96d8573458f8070e35b63c52f23` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:22.746000 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1608e39a-b92b-42d7-913f-491ee26d068c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a8af96d8573458f8070e35b63c52f23)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:22.752773 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1608e39a-b92b-42d7-913f-491ee26d068c admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:22.782239 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-f83b094c-0d5e-4a41-ab29-a2815a95ea0c None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:22.783367 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-f83b094c-0d5e-4a41-ab29-a2815a95ea0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:22.812937 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 846/1709] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:50:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:22.813900 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83b094c-0d5e-4a41-ab29-a2815a95ea0c None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:22.836879 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83b094c-0d5e-4a41-ab29-a2815a95ea0c None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:22.844229 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd7d534-eb8d-4919-b3c0-a2b428af8b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:22.845005 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd7d534-eb8d-4919-b3c0-a2b428af8b63 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:22.845408 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd7d534-eb8d-4919-b3c0-a2b428af8b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:22.845801 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd7d534-eb8d-4919-b3c0-a2b428af8b63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:22.846871 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd7d534-eb8d-4919-b3c0-a2b428af8b63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:22.849564 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd7d534-eb8d-4919-b3c0-a2b428af8b63 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:22.865273 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 847/1710] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:50:22 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:50:22.870868 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d70e0d-3e65-4e7e-a053-82974b065a02 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:22.871165 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d70e0d-3e65-4e7e-a053-82974b065a02 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:22.871564 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d70e0d-3e65-4e7e-a053-82974b065a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:22.902714 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-81d70e0d-3e65-4e7e-a053-82974b065a02 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:22.904193 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-81d70e0d-3e65-4e7e-a053-82974b065a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:22.920410 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 848/1711] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:50:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:22.920410 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d70e0d-3e65-4e7e-a053-82974b065a02 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:22.936625 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d70e0d-3e65-4e7e-a053-82974b065a02 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:22.947182 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236bc55c-bb74-4b8d-9a4e-5ad91b45f4b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:22.949137 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236bc55c-bb74-4b8d-9a4e-5ad91b45f4b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:22.949960 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236bc55c-bb74-4b8d-9a4e-5ad91b45f4b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:22.950730 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236bc55c-bb74-4b8d-9a4e-5ad91b45f4b3 admin admin] PATH_INFO: `/v3/projects/3ace515628524962a9adad597c499da2/users/be0172fefaab46e79f9d65f24cecefbf/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:22.955283 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:22.974523 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236bc55c-bb74-4b8d-9a4e-5ad91b45f4b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ace515628524962a9adad597c499da2, user_id=be0172fefaab46e79f9d65f24cecefbf, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:22.978870 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236bc55c-bb74-4b8d-9a4e-5ad91b45f4b3 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:23.035673 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-1608e39a-b92b-42d7-913f-491ee26d068c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a8af96d8573458f8070e35b63c52f23'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:23.035673 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-1608e39a-b92b-42d7-913f-491ee26d068c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a8af96d8573458f8070e35b63c52f23'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:23.036591 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-1608e39a-b92b-42d7-913f-491ee26d068c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a8af96d8573458f8070e35b63c52f23'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:23.055977 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 849/1712] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:50:22 2022] PUT /identity/v3/projects/3ace515628524962a9adad597c499da2/users/be0172fefaab46e79f9d65f24cecefbf/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:23.058081 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236bc55c-bb74-4b8d-9a4e-5ad91b45f4b3 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:23.064117 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-236bc55c-bb74-4b8d-9a4e-5ad91b45f4b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:23.139757 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-1608e39a-b92b-42d7-913f-491ee26d068c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a8af96d8573458f8070e35b63c52f23'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:23.190066 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-1608e39a-b92b-42d7-913f-491ee26d068c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a8af96d8573458f8070e35b63c52f23'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:23.251146 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 864/1713] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:50:22 2022] DELETE /identity/v3/users/7a8af96d8573458f8070e35b63c52f23 => generated 0 bytes in 619 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:23.259062 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85247292-2d3d-4536-a7a4-8673f44e86f5 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:23.259976 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85247292-2d3d-4536-a7a4-8673f44e86f5 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:23.260491 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85247292-2d3d-4536-a7a4-8673f44e86f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:23.274700 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236bc55c-bb74-4b8d-9a4e-5ad91b45f4b3 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:23.413484 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61581b7-6478-48cd-a8e8-9fe0707eb9ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:23.414493 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61581b7-6478-48cd-a8e8-9fe0707eb9ad admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:23.414493 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61581b7-6478-48cd-a8e8-9fe0707eb9ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:23.417777 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61581b7-6478-48cd-a8e8-9fe0707eb9ad admin admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:23.417777 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61581b7-6478-48cd-a8e8-9fe0707eb9ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:50:23.417777 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61581b7-6478-48cd-a8e8-9fe0707eb9ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:23.418395 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-85247292-2d3d-4536-a7a4-8673f44e86f5 None None] MFA Rules not processed for user `be0172fefaab46e79f9d65f24cecefbf`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:23.418570 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61581b7-6478-48cd-a8e8-9fe0707eb9ad admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:23.427516 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-85247292-2d3d-4536-a7a4-8673f44e86f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:23.436588 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 850/1714] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jun 15 14:50:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:50:23.440959 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61581b7-6478-48cd-a8e8-9fe0707eb9ad admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:23.473586 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61581b7-6478-48cd-a8e8-9fe0707eb9ad admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:23.486892 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc9e98d-b011-4ac7-adfb-8a4f055b9c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:23.487590 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc9e98d-b011-4ac7-adfb-8a4f055b9c95 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:23.487878 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc9e98d-b011-4ac7-adfb-8a4f055b9c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:23.488193 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc9e98d-b011-4ac7-adfb-8a4f055b9c95 admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:23.497248 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc9e98d-b011-4ac7-adfb-8a4f055b9c95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:23.523542 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc9e98d-b011-4ac7-adfb-8a4f055b9c95 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:23.587804 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:23.676362 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 865/1715] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:50:23 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 421 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:23.677556 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85247292-2d3d-4536-a7a4-8673f44e86f5 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:23.679403 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-85247292-2d3d-4536-a7a4-8673f44e86f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:23.726994 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85247292-2d3d-4536-a7a4-8673f44e86f5 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:23.733755 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 851/1716] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:50:23 2022] POST /identity/v3/projects => generated 383 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:50:23.738207 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc9e98d-b011-4ac7-adfb-8a4f055b9c95 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:23.828232 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f298b3f-8def-4d4b-8bdd-a38cb93c3797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:23.832969 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f298b3f-8def-4d4b-8bdd-a38cb93c3797 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:23.832969 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f298b3f-8def-4d4b-8bdd-a38cb93c3797 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:23.832969 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f298b3f-8def-4d4b-8bdd-a38cb93c3797 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:23.832969 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f298b3f-8def-4d4b-8bdd-a38cb93c3797 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:23.834440 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f298b3f-8def-4d4b-8bdd-a38cb93c3797 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:23.908831 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc9e98d-b011-4ac7-adfb-8a4f055b9c95 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:23.918773 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaea364-e28d-4e92-bf92-a871652ab914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:23.919738 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaea364-e28d-4e92-bf92-a871652ab914 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:23.920189 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaea364-e28d-4e92-bf92-a871652ab914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:23.920732 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaea364-e28d-4e92-bf92-a871652ab914 admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:23.921644 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaea364-e28d-4e92-bf92-a871652ab914 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:23.924194 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaea364-e28d-4e92-bf92-a871652ab914 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:23.941798 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:23.975775 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 866/1717] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:50:23 2022] POST /identity/v3/projects => generated 357 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:50:23.980990 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f298b3f-8def-4d4b-8bdd-a38cb93c3797 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:24.033945 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 852/1718] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:50:23 2022] POST /identity/v3/users => generated 292 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:50:24.037401 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaea364-e28d-4e92-bf92-a871652ab914 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:24.065466 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaea364-e28d-4e92-bf92-a871652ab914 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:24.141102 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f298b3f-8def-4d4b-8bdd-a38cb93c3797 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:24.155768 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad5348d-4545-48bb-8c09-1716bb8b16e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:24.157061 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad5348d-4545-48bb-8c09-1716bb8b16e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:24.157541 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad5348d-4545-48bb-8c09-1716bb8b16e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:24.158154 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad5348d-4545-48bb-8c09-1716bb8b16e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:24.158883 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad5348d-4545-48bb-8c09-1716bb8b16e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:24.160892 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad5348d-4545-48bb-8c09-1716bb8b16e6 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:24.209271 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a31eb1c-1be0-4a1d-9a27-2ced0fd9924f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:24.211005 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a31eb1c-1be0-4a1d-9a27-2ced0fd9924f admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:24.211005 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a31eb1c-1be0-4a1d-9a27-2ced0fd9924f admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:24.211170 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a31eb1c-1be0-4a1d-9a27-2ced0fd9924f admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:24.212003 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a31eb1c-1be0-4a1d-9a27-2ced0fd9924f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:24.213580 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a31eb1c-1be0-4a1d-9a27-2ced0fd9924f admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:24.244287 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 853/1719] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:50:24 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:50:24.273491 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a31eb1c-1be0-4a1d-9a27-2ced0fd9924f admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:24.295490 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 867/1720] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:50:23 2022] POST /identity/v3/users => generated 285 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:50:24.299920 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad5348d-4545-48bb-8c09-1716bb8b16e6 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:24.312663 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a31eb1c-1be0-4a1d-9a27-2ced0fd9924f admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:24.323245 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94ddddc-5183-4e48-8f82-b1168c7dd302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:24.324775 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94ddddc-5183-4e48-8f82-b1168c7dd302 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:24.324775 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94ddddc-5183-4e48-8f82-b1168c7dd302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:24.324954 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94ddddc-5183-4e48-8f82-b1168c7dd302 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:24.331186 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94ddddc-5183-4e48-8f82-b1168c7dd302 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:24.334176 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94ddddc-5183-4e48-8f82-b1168c7dd302 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:24.342568 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad5348d-4545-48bb-8c09-1716bb8b16e6 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:24.348793 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 854/1721] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:50:24 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:50:24.362264 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94ddddc-5183-4e48-8f82-b1168c7dd302 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:24.366566 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9f7131-c818-4825-b6f0-8775e03632c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:24.367643 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9f7131-c818-4825-b6f0-8775e03632c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:24.368063 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9f7131-c818-4825-b6f0-8775e03632c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:24.368465 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9f7131-c818-4825-b6f0-8775e03632c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:24.369209 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9f7131-c818-4825-b6f0-8775e03632c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:24.371200 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9f7131-c818-4825-b6f0-8775e03632c5 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:24.391957 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94ddddc-5183-4e48-8f82-b1168c7dd302 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:24.400756 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 868/1722] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:50:24 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:50:24.401284 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c2aff3-9bd7-43d7-83f2-cb19db8f7730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:24.402536 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c2aff3-9bd7-43d7-83f2-cb19db8f7730 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:24.403095 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c2aff3-9bd7-43d7-83f2-cb19db8f7730 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:24.403453 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c2aff3-9bd7-43d7-83f2-cb19db8f7730 admin admin] PATH_INFO: `/v3/projects/5f749af6d5204100a2a3434f1236c624/users/d9af53b1d8f34d73a7d05779a7bb8ddb/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:24.407116 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9f7131-c818-4825-b6f0-8775e03632c5 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:24.427966 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9f7131-c818-4825-b6f0-8775e03632c5 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:24.433174 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c2aff3-9bd7-43d7-83f2-cb19db8f7730 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f749af6d5204100a2a3434f1236c624, user_id=d9af53b1d8f34d73a7d05779a7bb8ddb, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:24.437695 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f861c10-67ec-4d2d-9137-34fed3c7e610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:24.438638 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c2aff3-9bd7-43d7-83f2-cb19db8f7730 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:24.439769 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f861c10-67ec-4d2d-9137-34fed3c7e610 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:24.440551 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f861c10-67ec-4d2d-9137-34fed3c7e610 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:24.441188 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f861c10-67ec-4d2d-9137-34fed3c7e610 admin admin] PATH_INFO: `/v3/projects/6cd6878b37194d2fa2116be9fe9da3c7/users/da86d49a129945c49e94e40fc93d01f9/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:24.473643 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f861c10-67ec-4d2d-9137-34fed3c7e610 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cd6878b37194d2fa2116be9fe9da3c7, user_id=da86d49a129945c49e94e40fc93d01f9, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:24.481133 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f861c10-67ec-4d2d-9137-34fed3c7e610 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:24.589211 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 869/1723] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:50:24 2022] PUT /identity/v3/projects/6cd6878b37194d2fa2116be9fe9da3c7/users/da86d49a129945c49e94e40fc93d01f9/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:50:24.594577 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f861c10-67ec-4d2d-9137-34fed3c7e610 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:24.597840 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 855/1724] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:50:24 2022] PUT /identity/v3/projects/5f749af6d5204100a2a3434f1236c624/users/d9af53b1d8f34d73a7d05779a7bb8ddb/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:24.613184 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12f3e78-95a0-4a4f-8447-450fa827e9ca None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:24.613612 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12f3e78-95a0-4a4f-8447-450fa827e9ca None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:24.614014 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12f3e78-95a0-4a4f-8447-450fa827e9ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:24.636897 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:24.673238 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-f12f3e78-95a0-4a4f-8447-450fa827e9ca None None] MFA Rules not processed for user `d9af53b1d8f34d73a7d05779a7bb8ddb`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:24.674509 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-f12f3e78-95a0-4a4f-8447-450fa827e9ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:24.722239 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f861c10-67ec-4d2d-9137-34fed3c7e610 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:24.789864 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 856/1725] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:50:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:50:24.791501 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12f3e78-95a0-4a4f-8447-450fa827e9ca None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:24.793901 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-f12f3e78-95a0-4a4f-8447-450fa827e9ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:24.871748 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:24.909213 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12f3e78-95a0-4a4f-8447-450fa827e9ca None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:24.917299 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36019501-183f-4c6a-bf80-0f09825d2d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:24.918204 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36019501-183f-4c6a-bf80-0f09825d2d4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:24.918599 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36019501-183f-4c6a-bf80-0f09825d2d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:24.918886 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36019501-183f-4c6a-bf80-0f09825d2d4c admin admin] PATH_INFO: `/v3/projects/b44b498a2c0743969db45edc6828faf6` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:24.920546 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36019501-183f-4c6a-bf80-0f09825d2d4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b44b498a2c0743969db45edc6828faf6)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:24.923550 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36019501-183f-4c6a-bf80-0f09825d2d4c admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:24.948222 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71389582-82c9-48ff-9b43-6343ead1cf6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:24.949834 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71389582-82c9-48ff-9b43-6343ead1cf6f admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:24.950492 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71389582-82c9-48ff-9b43-6343ead1cf6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:24.951166 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71389582-82c9-48ff-9b43-6343ead1cf6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:24.955025 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71389582-82c9-48ff-9b43-6343ead1cf6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:24.961830 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71389582-82c9-48ff-9b43-6343ead1cf6f admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:25.010754 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 870/1726] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:50:24 2022] GET /identity/v3/roles => generated 1420 bytes in 414 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:50:25.010754 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f2aa28-d3b9-4ed9-85a0-bbede1749217 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:25.010754 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f2aa28-d3b9-4ed9-85a0-bbede1749217 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:25.010754 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f2aa28-d3b9-4ed9-85a0-bbede1749217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:25.069883 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-36019501-183f-4c6a-bf80-0f09825d2d4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b44b498a2c0743969db45edc6828faf6', 'request_id': 'req-36019501-183f-4c6a-bf80-0f09825d2d4c'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:25.078014 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-b1f2aa28-d3b9-4ed9-85a0-bbede1749217 None None] MFA Rules not processed for user `ee9df934d1cd461abbf9c3cd90a8fdda`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:25.082564 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-b1f2aa28-d3b9-4ed9-85a0-bbede1749217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:25.180628 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-36019501-183f-4c6a-bf80-0f09825d2d4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b44b498a2c0743969db45edc6828faf6', 'request_id': 'req-36019501-183f-4c6a-bf80-0f09825d2d4c'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:25.196649 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-36019501-183f-4c6a-bf80-0f09825d2d4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b44b498a2c0743969db45edc6828faf6', 'request_id': 'req-36019501-183f-4c6a-bf80-0f09825d2d4c'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:25.250110 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-36019501-183f-4c6a-bf80-0f09825d2d4c admin admin] The token cache is being invalidate because project b44b498a2c0743969db45edc6828faf6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:50:25.251386 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 871/1727] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:50:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:25.252563 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f2aa28-d3b9-4ed9-85a0-bbede1749217 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:25.257692 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-36019501-183f-4c6a-bf80-0f09825d2d4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:25.258471 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-36019501-183f-4c6a-bf80-0f09825d2d4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:25.264438 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 857/1728] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:50:24 2022] DELETE /identity/v3/projects/b44b498a2c0743969db45edc6828faf6 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:25.265002 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36019501-183f-4c6a-bf80-0f09825d2d4c admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:25.269458 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-36019501-183f-4c6a-bf80-0f09825d2d4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:25.440838 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36019501-183f-4c6a-bf80-0f09825d2d4c admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:25.447068 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f2aa28-d3b9-4ed9-85a0-bbede1749217 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:25.451012 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2e44f6-a91c-47d8-bd73-d3fc61920ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:25.453040 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2e44f6-a91c-47d8-bd73-d3fc61920ddd admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:25.453902 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2e44f6-a91c-47d8-bd73-d3fc61920ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:25.454063 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc2203c-4060-4fc9-a851-c49528204c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:25.454726 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2e44f6-a91c-47d8-bd73-d3fc61920ddd admin admin] PATH_INFO: `/v3/projects/6cd6878b37194d2fa2116be9fe9da3c7/users/da86d49a129945c49e94e40fc93d01f9/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:25.455623 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc2203c-4060-4fc9-a851-c49528204c9e admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:25.456036 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc2203c-4060-4fc9-a851-c49528204c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:25.456568 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc2203c-4060-4fc9-a851-c49528204c9e admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:25.459771 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2e44f6-a91c-47d8-bd73-d3fc61920ddd admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cd6878b37194d2fa2116be9fe9da3c7, user_id=da86d49a129945c49e94e40fc93d01f9, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:25.466109 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2e44f6-a91c-47d8-bd73-d3fc61920ddd admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:25.467005 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc2203c-4060-4fc9-a851-c49528204c9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:25.472730 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc2203c-4060-4fc9-a851-c49528204c9e admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:25.478411 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:25.500220 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 858/1729] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:50:25 2022] PUT /identity/v3/projects/6cd6878b37194d2fa2116be9fe9da3c7/users/da86d49a129945c49e94e40fc93d01f9/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:25.504684 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d0d47-38e6-4ffc-8aea-7ff3444c0738 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:25.505413 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d0d47-38e6-4ffc-8aea-7ff3444c0738 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:25.507852 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d0d47-38e6-4ffc-8aea-7ff3444c0738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:25.610955 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 872/1730] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:50:25 2022] POST /identity/v3/projects => generated 383 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:50:25.613773 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1ae214-d1ac-4a31-9ded-07e00a27e84d None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:25.614296 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1ae214-d1ac-4a31-9ded-07e00a27e84d None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:25.614831 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1ae214-d1ac-4a31-9ded-07e00a27e84d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:25.616928 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-454d0d47-38e6-4ffc-8aea-7ff3444c0738 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:25.619116 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-454d0d47-38e6-4ffc-8aea-7ff3444c0738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:25.649183 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-4f1ae214-d1ac-4a31-9ded-07e00a27e84d None None] MFA Rules not processed for user `0650eb65117a4e27908c4355bfc46c3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:25.655903 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-4f1ae214-d1ac-4a31-9ded-07e00a27e84d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:25.790785 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 859/1731] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:50:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:25.794634 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92cfb6-671a-402d-9c9a-2243640f5987 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:25.795001 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92cfb6-671a-402d-9c9a-2243640f5987 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:25.795341 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92cfb6-671a-402d-9c9a-2243640f5987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:25.817575 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:25.841757 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-1e92cfb6-671a-402d-9c9a-2243640f5987 None None] MFA Rules not processed for user `da86d49a129945c49e94e40fc93d01f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:25.843078 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 873/1732] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:50:25 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:25.843239 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-1e92cfb6-671a-402d-9c9a-2243640f5987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:25.844051 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1ae214-d1ac-4a31-9ded-07e00a27e84d None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:25.846097 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-4f1ae214-d1ac-4a31-9ded-07e00a27e84d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:25.877580 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1ae214-d1ac-4a31-9ded-07e00a27e84d None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:25.883753 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6d98d7-2821-409b-99bd-e2cec11544a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:25.884772 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6d98d7-2821-409b-99bd-e2cec11544a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:25.885135 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6d98d7-2821-409b-99bd-e2cec11544a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:25.885489 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6d98d7-2821-409b-99bd-e2cec11544a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:25.887011 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6d98d7-2821-409b-99bd-e2cec11544a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:25.889847 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6d98d7-2821-409b-99bd-e2cec11544a6 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:25.993027 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 874/1733] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:50:25 2022] POST /identity/v3/users => generated 298 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:50:25.993027 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1796000-cb2e-4f9b-a3a7-eb662c45435b None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:25.993027 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1796000-cb2e-4f9b-a3a7-eb662c45435b None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:25.993027 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1796000-cb2e-4f9b-a3a7-eb662c45435b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:26.002062 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 860/1734] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:50:25 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:26.003771 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053150b-3ab4-4dee-8132-e3f263290270 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:26.004163 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053150b-3ab4-4dee-8132-e3f263290270 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:26.004768 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053150b-3ab4-4dee-8132-e3f263290270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:26.035428 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-b053150b-3ab4-4dee-8132-e3f263290270 None None] MFA Rules not processed for user `c739b61c8a8b453e979bda720bafebb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:26.037805 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-b053150b-3ab4-4dee-8132-e3f263290270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:26.077997 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-c1796000-cb2e-4f9b-a3a7-eb662c45435b None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:26.080174 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c1796000-cb2e-4f9b-a3a7-eb662c45435b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:26.114199 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 875/1735] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:50:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:26.119534 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1796000-cb2e-4f9b-a3a7-eb662c45435b None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:26.133526 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1796000-cb2e-4f9b-a3a7-eb662c45435b None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:26.139285 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8388915-a899-48fa-9ed0-d976a00e3187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:26.140046 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8388915-a899-48fa-9ed0-d976a00e3187 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:26.140351 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8388915-a899-48fa-9ed0-d976a00e3187 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:26.140628 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8388915-a899-48fa-9ed0-d976a00e3187 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:26.141431 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8388915-a899-48fa-9ed0-d976a00e3187 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:26.142518 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 861/1736] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Jun 15 14:50:26 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:26.143496 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8388915-a899-48fa-9ed0-d976a00e3187 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:26.144441 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b053150b-3ab4-4dee-8132-e3f263290270 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:26.146789 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-b053150b-3ab4-4dee-8132-e3f263290270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:26.162941 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 876/1737] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:50:26 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:50:26.162941 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8388915-a899-48fa-9ed0-d976a00e3187 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:26.179121 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8388915-a899-48fa-9ed0-d976a00e3187 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:26.190611 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a77f2d4-12e3-46e8-aad9-5d6d54b663d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:26.192608 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a77f2d4-12e3-46e8-aad9-5d6d54b663d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:26.192608 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a77f2d4-12e3-46e8-aad9-5d6d54b663d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:26.194969 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a77f2d4-12e3-46e8-aad9-5d6d54b663d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:26.196698 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a77f2d4-12e3-46e8-aad9-5d6d54b663d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:50:26.196824 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a77f2d4-12e3-46e8-aad9-5d6d54b663d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:26.201118 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a77f2d4-12e3-46e8-aad9-5d6d54b663d0 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:26.242321 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 877/1738] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jun 15 14:50:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:50:26.243389 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a77f2d4-12e3-46e8-aad9-5d6d54b663d0 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:26.245510 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-1a77f2d4-12e3-46e8-aad9-5d6d54b663d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:26.269334 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a77f2d4-12e3-46e8-aad9-5d6d54b663d0 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:26.278069 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505d977a-5874-43d6-9488-39f677c7a1f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:26.285846 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505d977a-5874-43d6-9488-39f677c7a1f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:26.286038 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b053150b-3ab4-4dee-8132-e3f263290270 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:26.286165 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505d977a-5874-43d6-9488-39f677c7a1f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:26.286165 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505d977a-5874-43d6-9488-39f677c7a1f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:26.286747 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e854849-b4d8-4fc5-a1e2-f4c0b185e1a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:26.287864 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e854849-b4d8-4fc5-a1e2-f4c0b185e1a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:26.288413 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e854849-b4d8-4fc5-a1e2-f4c0b185e1a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:26.288912 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e854849-b4d8-4fc5-a1e2-f4c0b185e1a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:26.302359 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e854849-b4d8-4fc5-a1e2-f4c0b185e1a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:26.302530 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505d977a-5874-43d6-9488-39f677c7a1f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:26.304683 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e854849-b4d8-4fc5-a1e2-f4c0b185e1a5 service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:26.306541 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505d977a-5874-43d6-9488-39f677c7a1f9 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:26.318037 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 862/1739] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:50:26 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:50:26.319785 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e854849-b4d8-4fc5-a1e2-f4c0b185e1a5 service cinder] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:26.326419 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 878/1740] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:50:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:50:26.328337 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505d977a-5874-43d6-9488-39f677c7a1f9 service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:26.332824 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e854849-b4d8-4fc5-a1e2-f4c0b185e1a5 service cinder] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:26.338693 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5e45a1-0014-4893-9ec5-26fb18d76d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:26.339604 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e45a1-0014-4893-9ec5-26fb18d76d33 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:26.339958 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e45a1-0014-4893-9ec5-26fb18d76d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:26.340717 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e45a1-0014-4893-9ec5-26fb18d76d33 admin admin] PATH_INFO: `/v3/projects/da72f868efca44a3b1b3737aa4cc5c42/users/cb5fea5206ed48978148155ad783d4bb/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:26.358304 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505d977a-5874-43d6-9488-39f677c7a1f9 service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:26.363583 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5e45a1-0014-4893-9ec5-26fb18d76d33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da72f868efca44a3b1b3737aa4cc5c42, user_id=cb5fea5206ed48978148155ad783d4bb, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:26.368432 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa271911-2bdf-4b84-9010-ddd85fb0240c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:26.372477 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa271911-2bdf-4b84-9010-ddd85fb0240c admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:26.372477 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa271911-2bdf-4b84-9010-ddd85fb0240c admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:26.372477 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa271911-2bdf-4b84-9010-ddd85fb0240c admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:26.373024 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5e45a1-0014-4893-9ec5-26fb18d76d33 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:26.373115 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa271911-2bdf-4b84-9010-ddd85fb0240c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:26.379042 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa271911-2bdf-4b84-9010-ddd85fb0240c admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:26.450837 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 863/1741] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:50:26 2022] PUT /identity/v3/projects/da72f868efca44a3b1b3737aa4cc5c42/users/cb5fea5206ed48978148155ad783d4bb/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:26.455205 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5e45a1-0014-4893-9ec5-26fb18d76d33 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:26.510617 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 879/1742] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:50:26 2022] POST /identity/v3/projects => generated 381 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:50:26.517303 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa271911-2bdf-4b84-9010-ddd85fb0240c admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:26.528189 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:26.649049 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5e45a1-0014-4893-9ec5-26fb18d76d33 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:26.656610 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31972526-41c3-4f55-af6a-f536eb7ccd95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:26.657322 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31972526-41c3-4f55-af6a-f536eb7ccd95 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:26.657607 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31972526-41c3-4f55-af6a-f536eb7ccd95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:26.657890 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31972526-41c3-4f55-af6a-f536eb7ccd95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:26.658650 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31972526-41c3-4f55-af6a-f536eb7ccd95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:26.658970 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa271911-2bdf-4b84-9010-ddd85fb0240c admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:26.660417 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31972526-41c3-4f55-af6a-f536eb7ccd95 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:26.667782 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04e70db-b9ac-43da-8a32-34445415557a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:26.668975 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04e70db-b9ac-43da-8a32-34445415557a admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:26.669476 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04e70db-b9ac-43da-8a32-34445415557a admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:26.669978 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04e70db-b9ac-43da-8a32-34445415557a admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:26.671475 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04e70db-b9ac-43da-8a32-34445415557a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:26.676872 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 864/1743] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:50:26 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:50:26.676872 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31972526-41c3-4f55-af6a-f536eb7ccd95 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:26.677096 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04e70db-b9ac-43da-8a32-34445415557a admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:26.677655 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-31972526-41c3-4f55-af6a-f536eb7ccd95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:26.725255 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:26.806330 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31972526-41c3-4f55-af6a-f536eb7ccd95 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:26.810768 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 880/1744] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:50:26 2022] POST /identity/v3/users => generated 291 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:50:26.812064 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04e70db-b9ac-43da-8a32-34445415557a admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:26.817375 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69a7f24-d033-40c5-89f5-51e424c375a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:26.818683 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a7f24-d033-40c5-89f5-51e424c375a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:26.820737 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a7f24-d033-40c5-89f5-51e424c375a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:26.821192 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a7f24-d033-40c5-89f5-51e424c375a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:26.824218 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-a69a7f24-d033-40c5-89f5-51e424c375a8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:26.831684 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04e70db-b9ac-43da-8a32-34445415557a admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:26.851431 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475bfdb9-8d88-466d-9980-dcde169f9e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:26.852681 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475bfdb9-8d88-466d-9980-dcde169f9e13 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:26.853165 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475bfdb9-8d88-466d-9980-dcde169f9e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:26.853558 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475bfdb9-8d88-466d-9980-dcde169f9e13 admin admin] PATH_INFO: `/v3/projects/da72f868efca44a3b1b3737aa4cc5c42/users/cb5fea5206ed48978148155ad783d4bb/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:26.855871 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475bfdb9-8d88-466d-9980-dcde169f9e13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da72f868efca44a3b1b3737aa4cc5c42, user_id=cb5fea5206ed48978148155ad783d4bb, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:26.858861 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475bfdb9-8d88-466d-9980-dcde169f9e13 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:26.896881 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 881/1745] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:50:26 2022] PUT /identity/v3/projects/da72f868efca44a3b1b3737aa4cc5c42/users/cb5fea5206ed48978148155ad783d4bb/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:26.898513 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475bfdb9-8d88-466d-9980-dcde169f9e13 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:26.950491 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69a7f24-d033-40c5-89f5-51e424c375a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:26.953316 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69a7f24-d033-40c5-89f5-51e424c375a8 service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:26.973281 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 865/1746] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:50:26 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:50:26.974941 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5dd434-8c76-4b92-ad87-6697bcfbd06b None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:26.976184 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5dd434-8c76-4b92-ad87-6697bcfbd06b None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:26.976793 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5dd434-8c76-4b92-ad87-6697bcfbd06b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:27.043550 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-2d5dd434-8c76-4b92-ad87-6697bcfbd06b None None] MFA Rules not processed for user `cb5fea5206ed48978148155ad783d4bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:27.044841 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-2d5dd434-8c76-4b92-ad87-6697bcfbd06b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:27.075084 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475bfdb9-8d88-466d-9980-dcde169f9e13 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:27.081221 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87127837-db9f-43fa-8fe1-38ab399268c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:27.082560 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87127837-db9f-43fa-8fe1-38ab399268c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:27.082903 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87127837-db9f-43fa-8fe1-38ab399268c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:27.083346 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87127837-db9f-43fa-8fe1-38ab399268c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:27.084350 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87127837-db9f-43fa-8fe1-38ab399268c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:27.087683 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87127837-db9f-43fa-8fe1-38ab399268c6 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:27.107033 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 882/1747] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:50:26 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:50:27.121190 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87127837-db9f-43fa-8fe1-38ab399268c6 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:27.150652 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87127837-db9f-43fa-8fe1-38ab399268c6 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:27.151265 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:27.171658 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 866/1748] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:50:26 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:27.180297 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f53acbd-80d2-43f5-bf5f-304ef092aa21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:27.182017 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f53acbd-80d2-43f5-bf5f-304ef092aa21 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:27.182455 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f53acbd-80d2-43f5-bf5f-304ef092aa21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:27.182877 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f53acbd-80d2-43f5-bf5f-304ef092aa21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:27.183232 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5dd434-8c76-4b92-ad87-6697bcfbd06b None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:27.186191 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f53acbd-80d2-43f5-bf5f-304ef092aa21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:27.188743 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f53acbd-80d2-43f5-bf5f-304ef092aa21 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:27.224333 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 883/1749] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:50:27 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:50:27.225478 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f53acbd-80d2-43f5-bf5f-304ef092aa21 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:27.248256 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f53acbd-80d2-43f5-bf5f-304ef092aa21 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:27.257954 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6eed0b-1700-41cc-99f3-04aff5333812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:27.259945 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6eed0b-1700-41cc-99f3-04aff5333812 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:27.260475 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6eed0b-1700-41cc-99f3-04aff5333812 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:27.260834 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6eed0b-1700-41cc-99f3-04aff5333812 admin admin] PATH_INFO: `/v3/projects/45d897d3d61c4767959f1f6a33ba2694/users/5c4f59ffdc7249648369b34c33ecb43f/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:27.275243 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5dd434-8c76-4b92-ad87-6697bcfbd06b None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:27.282076 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97c2473-7b72-4c96-8dcb-d4a14fa61a09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:27.286281 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c2473-7b72-4c96-8dcb-d4a14fa61a09 service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:27.286281 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c2473-7b72-4c96-8dcb-d4a14fa61a09 service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:27.286281 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c2473-7b72-4c96-8dcb-d4a14fa61a09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:27.286632 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6eed0b-1700-41cc-99f3-04aff5333812 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45d897d3d61c4767959f1f6a33ba2694, user_id=5c4f59ffdc7249648369b34c33ecb43f, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:27.290274 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6eed0b-1700-41cc-99f3-04aff5333812 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:27.297976 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97c2473-7b72-4c96-8dcb-d4a14fa61a09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:27.300128 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97c2473-7b72-4c96-8dcb-d4a14fa61a09 service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:27.312129 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 867/1750] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:50:27 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:50:27.313299 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9467ccc-666f-43ba-98cf-66e322b4df5f None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:27.313590 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9467ccc-666f-43ba-98cf-66e322b4df5f None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:27.315117 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9467ccc-666f-43ba-98cf-66e322b4df5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:27.347090 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-d9467ccc-666f-43ba-98cf-66e322b4df5f None None] MFA Rules not processed for user `19d82c04f9994654adfb10f4bd0573c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:27.348376 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-d9467ccc-666f-43ba-98cf-66e322b4df5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:27.415145 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 884/1751] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:50:27 2022] PUT /identity/v3/projects/45d897d3d61c4767959f1f6a33ba2694/users/5c4f59ffdc7249648369b34c33ecb43f/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:27.416441 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6eed0b-1700-41cc-99f3-04aff5333812 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:27.454643 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 868/1752] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Jun 15 14:50:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:27.466110 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa5d311-3617-41ee-b8c1-bc0af2629f63 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:27.466744 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa5d311-3617-41ee-b8c1-bc0af2629f63 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:27.467167 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa5d311-3617-41ee-b8c1-bc0af2629f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:27.563630 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-5fa5d311-3617-41ee-b8c1-bc0af2629f63 None None] MFA Rules not processed for user `5c4f59ffdc7249648369b34c33ecb43f`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:27.564901 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-5fa5d311-3617-41ee-b8c1-bc0af2629f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:27.618116 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6eed0b-1700-41cc-99f3-04aff5333812 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:27.634006 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c4e686-77a9-4079-9647-898078d4d2a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:27.637233 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c4e686-77a9-4079-9647-898078d4d2a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:27.637233 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c4e686-77a9-4079-9647-898078d4d2a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:27.637233 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c4e686-77a9-4079-9647-898078d4d2a5 admin admin] PATH_INFO: `/v3/users/d9af53b1d8f34d73a7d05779a7bb8ddb` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:27.642538 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c4e686-77a9-4079-9647-898078d4d2a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9af53b1d8f34d73a7d05779a7bb8ddb)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:27.645321 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c4e686-77a9-4079-9647-898078d4d2a5 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:27.770758 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 869/1753] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:50:27 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:27.777260 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa5d311-3617-41ee-b8c1-bc0af2629f63 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:27.799806 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa5d311-3617-41ee-b8c1-bc0af2629f63 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:27.809155 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b5cfc9-32d6-4ff0-8a8c-b3c35333b913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:27.809874 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b5cfc9-32d6-4ff0-8a8c-b3c35333b913 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:27.810129 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b5cfc9-32d6-4ff0-8a8c-b3c35333b913 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:27.810410 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b5cfc9-32d6-4ff0-8a8c-b3c35333b913 admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:27.811127 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b5cfc9-32d6-4ff0-8a8c-b3c35333b913 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:27.813126 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b5cfc9-32d6-4ff0-8a8c-b3c35333b913 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:27.863569 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:27.871447 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-76c4e686-77a9-4079-9647-898078d4d2a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9af53b1d8f34d73a7d05779a7bb8ddb'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:27.879071 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-76c4e686-77a9-4079-9647-898078d4d2a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9af53b1d8f34d73a7d05779a7bb8ddb'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:27.879071 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-76c4e686-77a9-4079-9647-898078d4d2a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9af53b1d8f34d73a7d05779a7bb8ddb'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:27.995374 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-76c4e686-77a9-4079-9647-898078d4d2a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9af53b1d8f34d73a7d05779a7bb8ddb'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:28.023249 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 870/1754] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:50:27 2022] POST /identity/v3/projects => generated 381 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:50:28.024506 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b5cfc9-32d6-4ff0-8a8c-b3c35333b913 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:28.029118 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-66b5cfc9-32d6-4ff0-8a8c-b3c35333b913 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:28.033353 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-76c4e686-77a9-4079-9647-898078d4d2a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9af53b1d8f34d73a7d05779a7bb8ddb'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:28.054866 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:28.120669 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 885/1755] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:50:27 2022] DELETE /identity/v3/users/d9af53b1d8f34d73a7d05779a7bb8ddb => generated 0 bytes in 705 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:28.125451 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c4e686-77a9-4079-9647-898078d4d2a5 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:28.129506 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-76c4e686-77a9-4079-9647-898078d4d2a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:28.225804 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b5cfc9-32d6-4ff0-8a8c-b3c35333b913 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:28.286106 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c4e686-77a9-4079-9647-898078d4d2a5 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:28.293127 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4d1cfe-2291-4daf-97cb-f5ee339855cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:28.294887 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4d1cfe-2291-4daf-97cb-f5ee339855cf admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:28.295177 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4d1cfe-2291-4daf-97cb-f5ee339855cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:28.295457 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4d1cfe-2291-4daf-97cb-f5ee339855cf admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:28.296202 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4d1cfe-2291-4daf-97cb-f5ee339855cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:28.298334 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4d1cfe-2291-4daf-97cb-f5ee339855cf admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:28.361726 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecd649d-1196-4cb2-8d35-efc9df8c3cb1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04dc5832333498e96a7e02bbfb9484c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:28.362690 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd649d-1196-4cb2-8d35-efc9df8c3cb1 service swift] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:28.363418 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd649d-1196-4cb2-8d35-efc9df8c3cb1 service swift] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:28.363872 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd649d-1196-4cb2-8d35-efc9df8c3cb1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:28.368199 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-cecd649d-1196-4cb2-8d35-efc9df8c3cb1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:28.389445 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 886/1756] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:50:28 2022] POST /identity/v3/users => generated 297 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:50:28.390828 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b67e56c-9908-4420-9113-5adae24fcea6 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:28.391117 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b67e56c-9908-4420-9113-5adae24fcea6 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:28.392781 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b67e56c-9908-4420-9113-5adae24fcea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:28.420841 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-1b67e56c-9908-4420-9113-5adae24fcea6 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:28.422104 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-1b67e56c-9908-4420-9113-5adae24fcea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:28.432924 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 887/1757] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:50:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:28.432924 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b67e56c-9908-4420-9113-5adae24fcea6 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:28.449403 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b67e56c-9908-4420-9113-5adae24fcea6 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:28.454218 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ed0786-568c-49d9-925b-e2b8b9885d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:28.454686 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecd649d-1196-4cb2-8d35-efc9df8c3cb1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:28.455464 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ed0786-568c-49d9-925b-e2b8b9885d3b admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:28.455464 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ed0786-568c-49d9-925b-e2b8b9885d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:28.455749 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ed0786-568c-49d9-925b-e2b8b9885d3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:28.456460 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ed0786-568c-49d9-925b-e2b8b9885d3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:28.456716 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecd649d-1196-4cb2-8d35-efc9df8c3cb1 service swift] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:28.459833 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ed0786-568c-49d9-925b-e2b8b9885d3b admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:28.468904 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 871/1758] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Jun 15 14:50:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 14:50:28.473234 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecd649d-1196-4cb2-8d35-efc9df8c3cb1 service swift] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:28.483359 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 888/1759] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:50:28 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:50:28.483637 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-cecd649d-1196-4cb2-8d35-efc9df8c3cb1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:28.484007 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ed0786-568c-49d9-925b-e2b8b9885d3b admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:28.518967 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ed0786-568c-49d9-925b-e2b8b9885d3b admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:28.551080 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf26aac-92de-4ff5-9940-b62c3c0f7638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:28.570534 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf26aac-92de-4ff5-9940-b62c3c0f7638 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:28.572553 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf26aac-92de-4ff5-9940-b62c3c0f7638 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:28.576876 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf26aac-92de-4ff5-9940-b62c3c0f7638 admin admin] PATH_INFO: `/v3/projects/315b456cd6f44cebaf0a9a920bb5da44/users/c69a17a612c644c0b4b4cdc7f7268059/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:28.610493 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf26aac-92de-4ff5-9940-b62c3c0f7638 admin admin] RBAC: Authorizing `identity:create_grant(project_id=315b456cd6f44cebaf0a9a920bb5da44, user_id=c69a17a612c644c0b4b4cdc7f7268059, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:28.614396 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf26aac-92de-4ff5-9940-b62c3c0f7638 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:28.640848 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecd649d-1196-4cb2-8d35-efc9df8c3cb1 service swift] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:28.647992 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec4a958-b5f7-40f8-ba73-e1492365c9d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:28.648910 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec4a958-b5f7-40f8-ba73-e1492365c9d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:28.649362 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec4a958-b5f7-40f8-ba73-e1492365c9d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:28.649835 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec4a958-b5f7-40f8-ba73-e1492365c9d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:28.699059 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec4a958-b5f7-40f8-ba73-e1492365c9d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:28.704939 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec4a958-b5f7-40f8-ba73-e1492365c9d3 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:28.713797 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:28.717920 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 889/1760] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:50:28 2022] PUT /identity/v3/projects/315b456cd6f44cebaf0a9a920bb5da44/users/c69a17a612c644c0b4b4cdc7f7268059/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:28.719049 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf26aac-92de-4ff5-9940-b62c3c0f7638 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:28.727867 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 872/1761] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:50:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:50:28.880563 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf26aac-92de-4ff5-9940-b62c3c0f7638 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:28.892507 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc52023-2e96-4117-9f2f-e9194040607a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:28.893413 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc52023-2e96-4117-9f2f-e9194040607a admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:28.893708 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc52023-2e96-4117-9f2f-e9194040607a admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:28.894142 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc52023-2e96-4117-9f2f-e9194040607a admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:28.895077 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc52023-2e96-4117-9f2f-e9194040607a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:28.896797 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc52023-2e96-4117-9f2f-e9194040607a admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:28.916184 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 890/1762] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:50:28 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:50:28.917634 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec4a958-b5f7-40f8-ba73-e1492365c9d3 service neutron] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:28.930958 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec4a958-b5f7-40f8-ba73-e1492365c9d3 service neutron] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:28.936515 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707b8d84-5571-45ec-a992-e0fdefcf69c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:28.937506 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b8d84-5571-45ec-a992-e0fdefcf69c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:28.937864 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b8d84-5571-45ec-a992-e0fdefcf69c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:28.938253 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b8d84-5571-45ec-a992-e0fdefcf69c6 admin admin] PATH_INFO: `/v3/projects/315b456cd6f44cebaf0a9a920bb5da44/users/c69a17a612c644c0b4b4cdc7f7268059/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:28.940475 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707b8d84-5571-45ec-a992-e0fdefcf69c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=315b456cd6f44cebaf0a9a920bb5da44, user_id=c69a17a612c644c0b4b4cdc7f7268059, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:28.943456 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707b8d84-5571-45ec-a992-e0fdefcf69c6 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:28.972147 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 873/1763] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:50:28 2022] PUT /identity/v3/projects/315b456cd6f44cebaf0a9a920bb5da44/users/c69a17a612c644c0b4b4cdc7f7268059/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:28.977728 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c61d98a-959f-41e3-a93d-55871801a215 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:28.983058 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c61d98a-959f-41e3-a93d-55871801a215 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:28.983058 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c61d98a-959f-41e3-a93d-55871801a215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:29.014362 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:29.032605 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-6c61d98a-959f-41e3-a93d-55871801a215 None None] MFA Rules not processed for user `c69a17a612c644c0b4b4cdc7f7268059`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:29.033848 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-6c61d98a-959f-41e3-a93d-55871801a215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:29.132209 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 891/1764] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:50:28 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:29.145281 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707b8d84-5571-45ec-a992-e0fdefcf69c6 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:29.148036 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-707b8d84-5571-45ec-a992-e0fdefcf69c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:29.280495 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707b8d84-5571-45ec-a992-e0fdefcf69c6 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:29.285035 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682fe65b-40ec-41c8-8653-3374f3b2c9ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:29.285791 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682fe65b-40ec-41c8-8653-3374f3b2c9ef service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:29.286145 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682fe65b-40ec-41c8-8653-3374f3b2c9ef service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:29.288715 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682fe65b-40ec-41c8-8653-3374f3b2c9ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:29.306306 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682fe65b-40ec-41c8-8653-3374f3b2c9ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:29.308556 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682fe65b-40ec-41c8-8653-3374f3b2c9ef service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:29.325597 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 874/1765] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:50:29 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:50:29.429566 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c61d98a-959f-41e3-a93d-55871801a215 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:29.431396 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-6c61d98a-959f-41e3-a93d-55871801a215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:29.523115 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c61d98a-959f-41e3-a93d-55871801a215 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:29.528484 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b367ca5-af1e-41ed-965c-cb49f3adcfb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:29.529381 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b367ca5-af1e-41ed-965c-cb49f3adcfb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:29.529661 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b367ca5-af1e-41ed-965c-cb49f3adcfb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:29.530028 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b367ca5-af1e-41ed-965c-cb49f3adcfb3 admin admin] PATH_INFO: `/v3/projects/5f749af6d5204100a2a3434f1236c624` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:29.531179 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b367ca5-af1e-41ed-965c-cb49f3adcfb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f749af6d5204100a2a3434f1236c624)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:29.533208 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b367ca5-af1e-41ed-965c-cb49f3adcfb3 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:29.622606 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682fe65b-40ec-41c8-8653-3374f3b2c9ef service cinder] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:29.631946 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682fe65b-40ec-41c8-8653-3374f3b2c9ef service cinder] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:29.635889 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23de0c5-d844-4432-b44e-52446a05bdb0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:29.636872 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23de0c5-d844-4432-b44e-52446a05bdb0 service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:29.637210 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23de0c5-d844-4432-b44e-52446a05bdb0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:29.637548 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23de0c5-d844-4432-b44e-52446a05bdb0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:29.655574 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-a23de0c5-d844-4432-b44e-52446a05bdb0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:29.702103 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23de0c5-d844-4432-b44e-52446a05bdb0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:29.703603 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23de0c5-d844-4432-b44e-52446a05bdb0 service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:29.714490 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 875/1766] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:50:29 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:50:29.830294 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-0b367ca5-af1e-41ed-965c-cb49f3adcfb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f749af6d5204100a2a3434f1236c624', 'request_id': 'req-0b367ca5-af1e-41ed-965c-cb49f3adcfb3'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:29.837490 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-0b367ca5-af1e-41ed-965c-cb49f3adcfb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f749af6d5204100a2a3434f1236c624', 'request_id': 'req-0b367ca5-af1e-41ed-965c-cb49f3adcfb3'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:29.946097 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-0b367ca5-af1e-41ed-965c-cb49f3adcfb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f749af6d5204100a2a3434f1236c624', 'request_id': 'req-0b367ca5-af1e-41ed-965c-cb49f3adcfb3'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:29.966034 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-0b367ca5-af1e-41ed-965c-cb49f3adcfb3 admin admin] The token cache is being invalidate because project 5f749af6d5204100a2a3434f1236c624 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:50:29.966576 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-0b367ca5-af1e-41ed-965c-cb49f3adcfb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:29.967388 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-0b367ca5-af1e-41ed-965c-cb49f3adcfb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:29.969319 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 892/1767] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:50:29 2022] DELETE /identity/v3/projects/5f749af6d5204100a2a3434f1236c624 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:50:29.973387 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23de0c5-d844-4432-b44e-52446a05bdb0 service cinder] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:29.988367 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23de0c5-d844-4432-b44e-52446a05bdb0 service cinder] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:30.055399 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82e53c1-2aa6-41d5-9bea-c17724e04e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:30.056368 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e53c1-2aa6-41d5-9bea-c17724e04e22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:30.056743 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e53c1-2aa6-41d5-9bea-c17724e04e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:30.057205 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e53c1-2aa6-41d5-9bea-c17724e04e22 admin admin] PATH_INFO: `/v3/users/cb5fea5206ed48978148155ad783d4bb` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:30.058929 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82e53c1-2aa6-41d5-9bea-c17724e04e22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb5fea5206ed48978148155ad783d4bb)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:30.061045 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82e53c1-2aa6-41d5-9bea-c17724e04e22 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:30.155011 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:30.281453 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-b82e53c1-2aa6-41d5-9bea-c17724e04e22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb5fea5206ed48978148155ad783d4bb'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:30.282288 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-b82e53c1-2aa6-41d5-9bea-c17724e04e22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb5fea5206ed48978148155ad783d4bb'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:30.294279 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-b82e53c1-2aa6-41d5-9bea-c17724e04e22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb5fea5206ed48978148155ad783d4bb'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:30.295296 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-b82e53c1-2aa6-41d5-9bea-c17724e04e22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb5fea5206ed48978148155ad783d4bb'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:30.370196 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-b82e53c1-2aa6-41d5-9bea-c17724e04e22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb5fea5206ed48978148155ad783d4bb'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:30.399001 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 876/1768] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:50:29 2022] DELETE /identity/v3/users/cb5fea5206ed48978148155ad783d4bb => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:30.907306 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b367ca5-af1e-41ed-965c-cb49f3adcfb3 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:30.908885 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-0b367ca5-af1e-41ed-965c-cb49f3adcfb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:30.925252 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b367ca5-af1e-41ed-965c-cb49f3adcfb3 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:30.948076 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:31.127546 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486b7b19-6892-4ffc-9c1d-ba193e913abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:31.128448 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486b7b19-6892-4ffc-9c1d-ba193e913abc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:31.128728 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486b7b19-6892-4ffc-9c1d-ba193e913abc admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:31.129001 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486b7b19-6892-4ffc-9c1d-ba193e913abc admin admin] PATH_INFO: `/v3/projects/da72f868efca44a3b1b3737aa4cc5c42` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:31.130231 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486b7b19-6892-4ffc-9c1d-ba193e913abc admin admin] RBAC: Authorizing `identity:delete_project(project_id=da72f868efca44a3b1b3737aa4cc5c42)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:31.132951 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486b7b19-6892-4ffc-9c1d-ba193e913abc admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:31.290409 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-486b7b19-6892-4ffc-9c1d-ba193e913abc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da72f868efca44a3b1b3737aa4cc5c42', 'request_id': 'req-486b7b19-6892-4ffc-9c1d-ba193e913abc'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:31.297979 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-486b7b19-6892-4ffc-9c1d-ba193e913abc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da72f868efca44a3b1b3737aa4cc5c42', 'request_id': 'req-486b7b19-6892-4ffc-9c1d-ba193e913abc'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:31.448185 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-486b7b19-6892-4ffc-9c1d-ba193e913abc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da72f868efca44a3b1b3737aa4cc5c42', 'request_id': 'req-486b7b19-6892-4ffc-9c1d-ba193e913abc'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:31.461553 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-486b7b19-6892-4ffc-9c1d-ba193e913abc admin admin] The token cache is being invalidate because project da72f868efca44a3b1b3737aa4cc5c42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:50:31.461892 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-486b7b19-6892-4ffc-9c1d-ba193e913abc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:31.462576 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-486b7b19-6892-4ffc-9c1d-ba193e913abc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:31.464166 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 893/1769] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:50:30 2022] DELETE /identity/v3/projects/da72f868efca44a3b1b3737aa4cc5c42 => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:31.470971 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600f526c-3907-4fd8-9396-03f77ce237fc None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:31.471480 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600f526c-3907-4fd8-9396-03f77ce237fc None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:31.471555 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600f526c-3907-4fd8-9396-03f77ce237fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:31.493778 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-600f526c-3907-4fd8-9396-03f77ce237fc None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:31.495394 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-600f526c-3907-4fd8-9396-03f77ce237fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:31.576645 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 877/1770] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:50:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:31.585533 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62860f52-7548-4b7c-bc77-28faf91bd40b None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:31.585936 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62860f52-7548-4b7c-bc77-28faf91bd40b None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:31.586303 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62860f52-7548-4b7c-bc77-28faf91bd40b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:31.605676 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-62860f52-7548-4b7c-bc77-28faf91bd40b None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:31.607012 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-62860f52-7548-4b7c-bc77-28faf91bd40b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:31.615223 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 894/1771] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:50:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:31.622079 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600f526c-3907-4fd8-9396-03f77ce237fc None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:31.633704 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600f526c-3907-4fd8-9396-03f77ce237fc None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:31.639339 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c9bb60-2927-45d0-84fe-32c431bfd3b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:31.640303 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c9bb60-2927-45d0-84fe-32c431bfd3b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:31.640485 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c9bb60-2927-45d0-84fe-32c431bfd3b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:31.640763 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c9bb60-2927-45d0-84fe-32c431bfd3b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:31.641663 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c9bb60-2927-45d0-84fe-32c431bfd3b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:50:31.641980 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c9bb60-2927-45d0-84fe-32c431bfd3b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:31.643640 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c9bb60-2927-45d0-84fe-32c431bfd3b0 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:31.661922 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 878/1772] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jun 15 14:50:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:50:31.666787 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62860f52-7548-4b7c-bc77-28faf91bd40b None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:31.679256 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62860f52-7548-4b7c-bc77-28faf91bd40b None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:31.683835 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bd70a4-d3f2-4632-82dc-7055d182b20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:31.684739 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd70a4-d3f2-4632-82dc-7055d182b20f admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:31.685524 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd70a4-d3f2-4632-82dc-7055d182b20f admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:31.686049 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd70a4-d3f2-4632-82dc-7055d182b20f admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:31.692691 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bd70a4-d3f2-4632-82dc-7055d182b20f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:31.695336 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bd70a4-d3f2-4632-82dc-7055d182b20f admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:31.826834 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 895/1773] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:50:31 2022] POST /identity/v3/projects => generated 377 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:50:31.834195 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c9bb60-2927-45d0-84fe-32c431bfd3b0 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:31.852116 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c9bb60-2927-45d0-84fe-32c431bfd3b0 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:31.931200 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919ea91e-5f42-408b-9dac-bae16259aa61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:31.932229 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919ea91e-5f42-408b-9dac-bae16259aa61 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:31.932739 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919ea91e-5f42-408b-9dac-bae16259aa61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:31.933181 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919ea91e-5f42-408b-9dac-bae16259aa61 admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:31.933914 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919ea91e-5f42-408b-9dac-bae16259aa61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:31.935890 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919ea91e-5f42-408b-9dac-bae16259aa61 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:32.120267 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 879/1774] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:50:31 2022] POST /identity/v3/users => generated 289 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:50:32.126080 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bd70a4-d3f2-4632-82dc-7055d182b20f admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:32.144265 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bd70a4-d3f2-4632-82dc-7055d182b20f admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:32.154390 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa5da7f-b458-41b2-acc5-904222ffe664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:32.155087 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5da7f-b458-41b2-acc5-904222ffe664 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:32.155368 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5da7f-b458-41b2-acc5-904222ffe664 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:32.155700 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5da7f-b458-41b2-acc5-904222ffe664 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:32.156383 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa5da7f-b458-41b2-acc5-904222ffe664 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:32.158052 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa5da7f-b458-41b2-acc5-904222ffe664 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:32.170025 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 896/1775] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:50:32 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:50:32.174078 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919ea91e-5f42-408b-9dac-bae16259aa61 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:32.185749 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919ea91e-5f42-408b-9dac-bae16259aa61 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:32.191212 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61c1233-1d3e-46cc-8271-fa79ab6f35f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:32.191903 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c1233-1d3e-46cc-8271-fa79ab6f35f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:32.192221 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c1233-1d3e-46cc-8271-fa79ab6f35f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:32.192504 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c1233-1d3e-46cc-8271-fa79ab6f35f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:32.193247 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61c1233-1d3e-46cc-8271-fa79ab6f35f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:32.195843 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61c1233-1d3e-46cc-8271-fa79ab6f35f6 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:32.209669 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 880/1776] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:50:32 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:50:32.213786 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa5da7f-b458-41b2-acc5-904222ffe664 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:32.223888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa5da7f-b458-41b2-acc5-904222ffe664 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:32.229176 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30a3962-27ea-4455-b56e-ab5627783d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:32.230102 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30a3962-27ea-4455-b56e-ab5627783d56 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:32.230377 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30a3962-27ea-4455-b56e-ab5627783d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:32.230613 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30a3962-27ea-4455-b56e-ab5627783d56 admin admin] PATH_INFO: `/v3/projects/138330dbc878490790d5ea4e9698839d/users/7c3894bcb09e4ac8ab0f074e2f2b7fad/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:32.248296 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30a3962-27ea-4455-b56e-ab5627783d56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=138330dbc878490790d5ea4e9698839d, user_id=7c3894bcb09e4ac8ab0f074e2f2b7fad, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:32.251017 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30a3962-27ea-4455-b56e-ab5627783d56 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:32.377555 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 897/1777] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:50:32 2022] PUT /identity/v3/projects/138330dbc878490790d5ea4e9698839d/users/7c3894bcb09e4ac8ab0f074e2f2b7fad/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:32.392401 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2171e6-5fc1-4f5c-b4f4-ed03965d6153 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:32.392691 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2171e6-5fc1-4f5c-b4f4-ed03965d6153 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:32.392971 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2171e6-5fc1-4f5c-b4f4-ed03965d6153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:32.434503 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:32.454843 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-bb2171e6-5fc1-4f5c-b4f4-ed03965d6153 None None] MFA Rules not processed for user `7c3894bcb09e4ac8ab0f074e2f2b7fad`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:32.460312 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-bb2171e6-5fc1-4f5c-b4f4-ed03965d6153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:32.549788 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 881/1778] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:50:32 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:32.557327 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30a3962-27ea-4455-b56e-ab5627783d56 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:32.656283 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:32.656283 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30a3962-27ea-4455-b56e-ab5627783d56 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:32.674570 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e7021a-9074-4fa0-922b-ee2995b21cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:32.674570 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e7021a-9074-4fa0-922b-ee2995b21cf7 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:32.674570 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e7021a-9074-4fa0-922b-ee2995b21cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:32.674570 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e7021a-9074-4fa0-922b-ee2995b21cf7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:32.675038 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e7021a-9074-4fa0-922b-ee2995b21cf7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:32.676904 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e7021a-9074-4fa0-922b-ee2995b21cf7 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:32.939854 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2171e6-5fc1-4f5c-b4f4-ed03965d6153 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:32.957328 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 898/1779] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:50:32 2022] POST /identity/v3/projects => generated 375 bytes in 397 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:50:32.970189 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2171e6-5fc1-4f5c-b4f4-ed03965d6153 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:33.083270 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99676dc-1fe5-452c-a043-67c5792bb5dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:33.084041 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99676dc-1fe5-452c-a043-67c5792bb5dc admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:33.084878 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99676dc-1fe5-452c-a043-67c5792bb5dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:33.085162 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99676dc-1fe5-452c-a043-67c5792bb5dc admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:33.086055 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99676dc-1fe5-452c-a043-67c5792bb5dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:33.089451 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99676dc-1fe5-452c-a043-67c5792bb5dc admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:33.221635 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 882/1780] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:50:32 2022] POST /identity/v3/users => generated 288 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:50:33.226009 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e7021a-9074-4fa0-922b-ee2995b21cf7 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:33.239168 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e7021a-9074-4fa0-922b-ee2995b21cf7 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:33.247874 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a0ced6-edae-46e6-b244-aed0cfa24416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:33.248593 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a0ced6-edae-46e6-b244-aed0cfa24416 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:33.248900 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a0ced6-edae-46e6-b244-aed0cfa24416 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:33.249185 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a0ced6-edae-46e6-b244-aed0cfa24416 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:33.249940 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a0ced6-edae-46e6-b244-aed0cfa24416 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:33.251612 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a0ced6-edae-46e6-b244-aed0cfa24416 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:33.267259 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 899/1781] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:50:33 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:50:33.272454 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99676dc-1fe5-452c-a043-67c5792bb5dc admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:33.283375 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99676dc-1fe5-452c-a043-67c5792bb5dc admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:33.293902 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86ffdf1-f2a8-4370-a252-936048f5d4be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:33.294603 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86ffdf1-f2a8-4370-a252-936048f5d4be admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:33.294897 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86ffdf1-f2a8-4370-a252-936048f5d4be admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:33.295172 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86ffdf1-f2a8-4370-a252-936048f5d4be admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:33.295921 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86ffdf1-f2a8-4370-a252-936048f5d4be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:33.297581 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86ffdf1-f2a8-4370-a252-936048f5d4be admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:33.309898 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 883/1782] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:50:33 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:50:33.314922 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a0ced6-edae-46e6-b244-aed0cfa24416 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:33.328968 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a0ced6-edae-46e6-b244-aed0cfa24416 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:33.335210 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e78a47d-0606-4667-8ab1-bbf762ca59e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:33.336144 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e78a47d-0606-4667-8ab1-bbf762ca59e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:33.337397 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e78a47d-0606-4667-8ab1-bbf762ca59e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:33.337397 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e78a47d-0606-4667-8ab1-bbf762ca59e8 admin admin] PATH_INFO: `/v3/projects/835dcbfb58e54ffab5b572f3b3abc245/users/7416610dce2c46ef849f64cd971c5d28/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:33.353815 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e78a47d-0606-4667-8ab1-bbf762ca59e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=835dcbfb58e54ffab5b572f3b3abc245, user_id=7416610dce2c46ef849f64cd971c5d28, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:33.357127 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e78a47d-0606-4667-8ab1-bbf762ca59e8 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:33.525217 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 900/1783] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:50:33 2022] PUT /identity/v3/projects/835dcbfb58e54ffab5b572f3b3abc245/users/7416610dce2c46ef849f64cd971c5d28/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:50:33.546553 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e90f8e-3689-472c-bb80-c6559e20e007 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:33.551108 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e90f8e-3689-472c-bb80-c6559e20e007 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:33.556040 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e90f8e-3689-472c-bb80-c6559e20e007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:33.597457 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-d3e90f8e-3689-472c-bb80-c6559e20e007 None None] MFA Rules not processed for user `7416610dce2c46ef849f64cd971c5d28`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:33.598927 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-d3e90f8e-3689-472c-bb80-c6559e20e007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:33.766853 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 884/1784] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:50:33 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:33.768601 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e78a47d-0606-4667-8ab1-bbf762ca59e8 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:33.933511 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e78a47d-0606-4667-8ab1-bbf762ca59e8 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:33.953305 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf83a4d-e26e-4abf-a76f-646e1bd77aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:33.954267 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf83a4d-e26e-4abf-a76f-646e1bd77aa2 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:33.954712 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf83a4d-e26e-4abf-a76f-646e1bd77aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:33.955361 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf83a4d-e26e-4abf-a76f-646e1bd77aa2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:33.956360 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf83a4d-e26e-4abf-a76f-646e1bd77aa2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:33.959757 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf83a4d-e26e-4abf-a76f-646e1bd77aa2 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:34.081397 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 901/1785] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:50:33 2022] POST /identity/v3/projects => generated 379 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:50:34.083316 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e90f8e-3689-472c-bb80-c6559e20e007 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:34.104956 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e90f8e-3689-472c-bb80-c6559e20e007 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:34.112006 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:34.144352 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a9dddc-0ab3-48e5-9d59-6760deede0b8 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:34.147294 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a9dddc-0ab3-48e5-9d59-6760deede0b8 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:34.147781 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a9dddc-0ab3-48e5-9d59-6760deede0b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:34.212775 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-04a9dddc-0ab3-48e5-9d59-6760deede0b8 None None] MFA Rules not processed for user `c739b61c8a8b453e979bda720bafebb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:34.217041 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-04a9dddc-0ab3-48e5-9d59-6760deede0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:34.291412 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5238da-6a34-49e6-97f8-927d1fb633a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:34.293509 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5238da-6a34-49e6-97f8-927d1fb633a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:34.294486 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5238da-6a34-49e6-97f8-927d1fb633a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:34.295431 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5238da-6a34-49e6-97f8-927d1fb633a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:34.297037 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5238da-6a34-49e6-97f8-927d1fb633a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:34.302993 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5238da-6a34-49e6-97f8-927d1fb633a9 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:34.362753 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 902/1786] 10.222.0.46 () {58 vars in 1015 bytes} [Wed Jun 15 14:50:34 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:50:34.406092 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a9dddc-0ab3-48e5-9d59-6760deede0b8 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:34.409686 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-04a9dddc-0ab3-48e5-9d59-6760deede0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:34.421746 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 885/1787] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:50:34 2022] POST /identity/v3/users => generated 296 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:50:34.426294 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5238da-6a34-49e6-97f8-927d1fb633a9 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:34.434106 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a9dddc-0ab3-48e5-9d59-6760deede0b8 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:34.440957 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c66c3a4-db50-4a0e-a258-964f5cc1fc1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:34.442006 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c66c3a4-db50-4a0e-a258-964f5cc1fc1f service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:34.446735 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c66c3a4-db50-4a0e-a258-964f5cc1fc1f service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:34.446735 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c66c3a4-db50-4a0e-a258-964f5cc1fc1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:34.446994 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5238da-6a34-49e6-97f8-927d1fb633a9 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:34.454495 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcd2412-3862-473d-8e57-84b92d0825c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:34.455487 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcd2412-3862-473d-8e57-84b92d0825c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:34.455898 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c66c3a4-db50-4a0e-a258-964f5cc1fc1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:34.456358 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcd2412-3862-473d-8e57-84b92d0825c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:34.457586 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c66c3a4-db50-4a0e-a258-964f5cc1fc1f service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:34.458065 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcd2412-3862-473d-8e57-84b92d0825c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:34.459176 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcd2412-3862-473d-8e57-84b92d0825c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:34.472180 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcd2412-3862-473d-8e57-84b92d0825c8 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:34.486274 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 903/1788] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:50:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:50:34.495408 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 886/1789] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:50:34 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:50:34.502154 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c66c3a4-db50-4a0e-a258-964f5cc1fc1f service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:34.524207 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c66c3a4-db50-4a0e-a258-964f5cc1fc1f service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:34.541836 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db7aa00-7128-44d4-b2a9-3d96979662ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:34.542835 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db7aa00-7128-44d4-b2a9-3d96979662ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:34.543216 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db7aa00-7128-44d4-b2a9-3d96979662ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:34.543599 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db7aa00-7128-44d4-b2a9-3d96979662ee admin admin] PATH_INFO: `/v3/projects/8c2abc2fdc4b43fbb3e54bf9ced6e3d8/users/2452d53ecab0465883487ac81183a31e/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:34.593827 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db7aa00-7128-44d4-b2a9-3d96979662ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c2abc2fdc4b43fbb3e54bf9ced6e3d8, user_id=2452d53ecab0465883487ac81183a31e, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:34.600221 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db7aa00-7128-44d4-b2a9-3d96979662ee admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:34.617533 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:34.786954 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 904/1790] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:50:34 2022] PUT /identity/v3/projects/8c2abc2fdc4b43fbb3e54bf9ced6e3d8/users/2452d53ecab0465883487ac81183a31e/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:34.792812 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcd2412-3862-473d-8e57-84b92d0825c8 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:34.807862 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcd2412-3862-473d-8e57-84b92d0825c8 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:34.975300 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1222658-ef12-44a2-84ee-0915d22ea456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:34.976009 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1222658-ef12-44a2-84ee-0915d22ea456 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:34.976324 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1222658-ef12-44a2-84ee-0915d22ea456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:34.976609 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1222658-ef12-44a2-84ee-0915d22ea456 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:34.985189 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1222658-ef12-44a2-84ee-0915d22ea456 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:34.985189 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1222658-ef12-44a2-84ee-0915d22ea456 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:35.011797 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 887/1791] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:50:34 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:50:35.016486 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db7aa00-7128-44d4-b2a9-3d96979662ee admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:35.054085 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db7aa00-7128-44d4-b2a9-3d96979662ee admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:35.063607 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25982377-e149-42a4-9089-fdc657b281ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:35.065511 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25982377-e149-42a4-9089-fdc657b281ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:35.066323 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25982377-e149-42a4-9089-fdc657b281ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:35.066614 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25982377-e149-42a4-9089-fdc657b281ae admin admin] PATH_INFO: `/v3/projects/8c2abc2fdc4b43fbb3e54bf9ced6e3d8/users/2452d53ecab0465883487ac81183a31e/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:35.071152 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25982377-e149-42a4-9089-fdc657b281ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c2abc2fdc4b43fbb3e54bf9ced6e3d8, user_id=2452d53ecab0465883487ac81183a31e, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:35.074923 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25982377-e149-42a4-9089-fdc657b281ae admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:35.107460 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 905/1792] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:50:35 2022] PUT /identity/v3/projects/8c2abc2fdc4b43fbb3e54bf9ced6e3d8/users/2452d53ecab0465883487ac81183a31e/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:35.120293 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b2c589-d3e6-41eb-be52-346a4446ada1 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:35.120602 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b2c589-d3e6-41eb-be52-346a4446ada1 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:35.120861 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b2c589-d3e6-41eb-be52-346a4446ada1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:35.157381 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-56b2c589-d3e6-41eb-be52-346a4446ada1 None None] MFA Rules not processed for user `2452d53ecab0465883487ac81183a31e`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:35.158681 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-56b2c589-d3e6-41eb-be52-346a4446ada1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:35.321264 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 888/1793] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:50:35 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:35.334266 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25982377-e149-42a4-9089-fdc657b281ae admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:35.336485 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-25982377-e149-42a4-9089-fdc657b281ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:35.435246 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25982377-e149-42a4-9089-fdc657b281ae admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:35.439900 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f56f36-7578-4167-95a3-f5bd5f86808f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:35.440825 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f56f36-7578-4167-95a3-f5bd5f86808f service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:35.441166 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f56f36-7578-4167-95a3-f5bd5f86808f service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:35.441572 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f56f36-7578-4167-95a3-f5bd5f86808f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:35.453120 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f56f36-7578-4167-95a3-f5bd5f86808f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:35.454751 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f56f36-7578-4167-95a3-f5bd5f86808f service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:35.476511 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 906/1794] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:50:35 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:50:35.682340 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b2c589-d3e6-41eb-be52-346a4446ada1 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:35.684139 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-56b2c589-d3e6-41eb-be52-346a4446ada1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:35.710955 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b2c589-d3e6-41eb-be52-346a4446ada1 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:35.722163 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3475042a-63fe-41ea-bceb-3dbd5c96e3e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:35.723127 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3475042a-63fe-41ea-bceb-3dbd5c96e3e1 service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:35.724551 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3475042a-63fe-41ea-bceb-3dbd5c96e3e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:35.725023 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3475042a-63fe-41ea-bceb-3dbd5c96e3e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:35.743771 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:35.885119 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3475042a-63fe-41ea-bceb-3dbd5c96e3e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:35.888154 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3475042a-63fe-41ea-bceb-3dbd5c96e3e1 service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:35.913256 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 889/1795] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:50:35 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:50:36.355097 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eff79d1-4072-4ab8-a62a-43db7403ecf0 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:36.355097 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eff79d1-4072-4ab8-a62a-43db7403ecf0 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:36.355097 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eff79d1-4072-4ab8-a62a-43db7403ecf0 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:36.355097 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 907/1796] 10.222.0.42 () {54 vars in 921 bytes} [Wed Jun 15 14:50:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:50:36.364539 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05548e08-8d3e-4f36-a926-ef86b9540b45 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:36.365314 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05548e08-8d3e-4f36-a926-ef86b9540b45 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:36.367363 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05548e08-8d3e-4f36-a926-ef86b9540b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:36.433383 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-05548e08-8d3e-4f36-a926-ef86b9540b45 None None] MFA Rules not processed for user `0927c05c340c48e48b18754888aa9c35`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:36.435809 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-05548e08-8d3e-4f36-a926-ef86b9540b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:36.565544 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 890/1797] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Jun 15 14:50:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:36.587657 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eff79d1-4072-4ab8-a62a-43db7403ecf0 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:36.591489 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-3eff79d1-4072-4ab8-a62a-43db7403ecf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:36.629640 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eff79d1-4072-4ab8-a62a-43db7403ecf0 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:36.639353 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c118b70-c117-4939-bafd-46c1674b0213 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0927c05c340c48e48b18754888aa9c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:36.640025 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c118b70-c117-4939-bafd-46c1674b0213 service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:36.640347 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c118b70-c117-4939-bafd-46c1674b0213 service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:36.640934 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c118b70-c117-4939-bafd-46c1674b0213 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:36.652852 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c118b70-c117-4939-bafd-46c1674b0213 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:36.655193 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c118b70-c117-4939-bafd-46c1674b0213 service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:36.672582 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 908/1798] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:50:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:50:46.545780 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=55 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:46.662814 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=39,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:51.066824 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d4c58d-1534-434f-9957-9dbaa9b97396 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:51.069657 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d4c58d-1534-434f-9957-9dbaa9b97396 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:51.076433 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d4c58d-1534-434f-9957-9dbaa9b97396 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:51.076433 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 891/1799] 10.222.0.42 () {54 vars in 921 bytes} [Wed Jun 15 14:50:51 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:50:51.081992 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc31cfb-bdd7-44dd-b319-459bf43bd408 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:51.082475 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc31cfb-bdd7-44dd-b319-459bf43bd408 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:51.082564 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc31cfb-bdd7-44dd-b319-459bf43bd408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:51.112813 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-0dc31cfb-bdd7-44dd-b319-459bf43bd408 None None] MFA Rules not processed for user `0927c05c340c48e48b18754888aa9c35`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:51.114388 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-0dc31cfb-bdd7-44dd-b319-459bf43bd408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:51.129906 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 909/1800] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Jun 15 14:50:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:51.231451 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d4c58d-1534-434f-9957-9dbaa9b97396 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:51.233444 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-c1d4c58d-1534-434f-9957-9dbaa9b97396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:51.266542 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d4c58d-1534-434f-9957-9dbaa9b97396 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:51.295181 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afe304a-8d9d-462a-ac61-06ce62ce5d80 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0927c05c340c48e48b18754888aa9c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:51.295908 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe304a-8d9d-462a-ac61-06ce62ce5d80 service nova] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:51.296259 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe304a-8d9d-462a-ac61-06ce62ce5d80 service nova] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:51.296540 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe304a-8d9d-462a-ac61-06ce62ce5d80 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:51.327303 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afe304a-8d9d-462a-ac61-06ce62ce5d80 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:51.328419 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afe304a-8d9d-462a-ac61-06ce62ce5d80 service nova] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:51.363824 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 892/1801] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:50:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:50:54.437904 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc31cfb-bdd7-44dd-b319-459bf43bd408 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:54.564690 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc31cfb-bdd7-44dd-b319-459bf43bd408 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:54.570289 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d50d7e7-1402-4342-a5e3-318d96e2a560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:54.571149 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d50d7e7-1402-4342-a5e3-318d96e2a560 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:54.571564 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d50d7e7-1402-4342-a5e3-318d96e2a560 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:54.571850 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d50d7e7-1402-4342-a5e3-318d96e2a560 admin admin] PATH_INFO: `/v3/users/7c3894bcb09e4ac8ab0f074e2f2b7fad` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:54.573312 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d50d7e7-1402-4342-a5e3-318d96e2a560 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c3894bcb09e4ac8ab0f074e2f2b7fad)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:54.575840 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d50d7e7-1402-4342-a5e3-318d96e2a560 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:54.795364 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-0d50d7e7-1402-4342-a5e3-318d96e2a560 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c3894bcb09e4ac8ab0f074e2f2b7fad'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:54.796030 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-0d50d7e7-1402-4342-a5e3-318d96e2a560 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c3894bcb09e4ac8ab0f074e2f2b7fad'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:54.796677 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-0d50d7e7-1402-4342-a5e3-318d96e2a560 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c3894bcb09e4ac8ab0f074e2f2b7fad'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:54.911216 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-0d50d7e7-1402-4342-a5e3-318d96e2a560 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c3894bcb09e4ac8ab0f074e2f2b7fad'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:54.926626 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-0d50d7e7-1402-4342-a5e3-318d96e2a560 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c3894bcb09e4ac8ab0f074e2f2b7fad'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:54.940781 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 910/1802] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:50:54 2022] DELETE /identity/v3/users/7c3894bcb09e4ac8ab0f074e2f2b7fad => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:54.946564 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f8d355-e6e5-4990-b544-0b95221f45b3 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:54.947146 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f8d355-e6e5-4990-b544-0b95221f45b3 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:54.947283 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f8d355-e6e5-4990-b544-0b95221f45b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:54.968372 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-68f8d355-e6e5-4990-b544-0b95221f45b3 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:54.969634 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-68f8d355-e6e5-4990-b544-0b95221f45b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:55.059475 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 893/1803] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:50:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:55.069611 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d50d7e7-1402-4342-a5e3-318d96e2a560 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:55.071944 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-0d50d7e7-1402-4342-a5e3-318d96e2a560 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:55.136771 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d50d7e7-1402-4342-a5e3-318d96e2a560 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:55.140686 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cbde59-bd2c-4eec-b4b6-0ac251d086be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:55.141428 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cbde59-bd2c-4eec-b4b6-0ac251d086be service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:55.141712 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cbde59-bd2c-4eec-b4b6-0ac251d086be service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:55.141997 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cbde59-bd2c-4eec-b4b6-0ac251d086be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:55.153452 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cbde59-bd2c-4eec-b4b6-0ac251d086be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:55.154937 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cbde59-bd2c-4eec-b4b6-0ac251d086be service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:55.165450 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 911/1804] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:50:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:50:55.696193 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f8d355-e6e5-4990-b544-0b95221f45b3 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:55.704533 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-68f8d355-e6e5-4990-b544-0b95221f45b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:55.747132 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f8d355-e6e5-4990-b544-0b95221f45b3 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:55.755123 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c641a3b8-9df9-42bf-be38-46fbf9c5535c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:55.756300 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c641a3b8-9df9-42bf-be38-46fbf9c5535c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:55.756591 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c641a3b8-9df9-42bf-be38-46fbf9c5535c admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:55.756874 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c641a3b8-9df9-42bf-be38-46fbf9c5535c admin admin] PATH_INFO: `/v3/projects/138330dbc878490790d5ea4e9698839d` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:55.758235 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c641a3b8-9df9-42bf-be38-46fbf9c5535c admin admin] RBAC: Authorizing `identity:delete_project(project_id=138330dbc878490790d5ea4e9698839d)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:55.760542 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c641a3b8-9df9-42bf-be38-46fbf9c5535c admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:55.916410 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-c641a3b8-9df9-42bf-be38-46fbf9c5535c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '138330dbc878490790d5ea4e9698839d', 'request_id': 'req-c641a3b8-9df9-42bf-be38-46fbf9c5535c'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:55.991134 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-c641a3b8-9df9-42bf-be38-46fbf9c5535c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '138330dbc878490790d5ea4e9698839d', 'request_id': 'req-c641a3b8-9df9-42bf-be38-46fbf9c5535c'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:55.998747 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-c641a3b8-9df9-42bf-be38-46fbf9c5535c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '138330dbc878490790d5ea4e9698839d', 'request_id': 'req-c641a3b8-9df9-42bf-be38-46fbf9c5535c'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:56.010471 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-c641a3b8-9df9-42bf-be38-46fbf9c5535c admin admin] The token cache is being invalidate because project 138330dbc878490790d5ea4e9698839d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:50:56.013674 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-c641a3b8-9df9-42bf-be38-46fbf9c5535c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:56.013674 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-c641a3b8-9df9-42bf-be38-46fbf9c5535c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:56.013674 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 894/1805] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:50:55 2022] DELETE /identity/v3/projects/138330dbc878490790d5ea4e9698839d => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:56.017534 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cbde59-bd2c-4eec-b4b6-0ac251d086be service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:56.028883 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cbde59-bd2c-4eec-b4b6-0ac251d086be service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:56.051159 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:56.098658 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6cd87b-5486-4e02-8406-636c4ee753f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:56.099637 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6cd87b-5486-4e02-8406-636c4ee753f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:56.099938 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6cd87b-5486-4e02-8406-636c4ee753f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:56.100263 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6cd87b-5486-4e02-8406-636c4ee753f7 admin admin] PATH_INFO: `/v3/users/7416610dce2c46ef849f64cd971c5d28` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:56.102301 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6cd87b-5486-4e02-8406-636c4ee753f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7416610dce2c46ef849f64cd971c5d28)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:56.104656 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6cd87b-5486-4e02-8406-636c4ee753f7 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:56.378605 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ce6cd87b-5486-4e02-8406-636c4ee753f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7416610dce2c46ef849f64cd971c5d28'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:56.383379 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ce6cd87b-5486-4e02-8406-636c4ee753f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7416610dce2c46ef849f64cd971c5d28'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:56.383379 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ce6cd87b-5486-4e02-8406-636c4ee753f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7416610dce2c46ef849f64cd971c5d28'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:56.471362 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ce6cd87b-5486-4e02-8406-636c4ee753f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7416610dce2c46ef849f64cd971c5d28'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:56.485109 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ce6cd87b-5486-4e02-8406-636c4ee753f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7416610dce2c46ef849f64cd971c5d28'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:56.516997 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 912/1806] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:50:56 2022] DELETE /identity/v3/users/7416610dce2c46ef849f64cd971c5d28 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:57.137059 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c641a3b8-9df9-42bf-be38-46fbf9c5535c admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:57.138526 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-c641a3b8-9df9-42bf-be38-46fbf9c5535c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:57.153124 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c641a3b8-9df9-42bf-be38-46fbf9c5535c admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:57.227128 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eee27a8-a1ae-4eed-b50f-81d1892ab0fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:57.228405 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee27a8-a1ae-4eed-b50f-81d1892ab0fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:57.228696 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee27a8-a1ae-4eed-b50f-81d1892ab0fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:57.228996 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee27a8-a1ae-4eed-b50f-81d1892ab0fd admin admin] PATH_INFO: `/v3/projects/835dcbfb58e54ffab5b572f3b3abc245` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:57.230577 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eee27a8-a1ae-4eed-b50f-81d1892ab0fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=835dcbfb58e54ffab5b572f3b3abc245)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:57.232531 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eee27a8-a1ae-4eed-b50f-81d1892ab0fd admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:57.245267 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:57.356333 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-2eee27a8-a1ae-4eed-b50f-81d1892ab0fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '835dcbfb58e54ffab5b572f3b3abc245', 'request_id': 'req-2eee27a8-a1ae-4eed-b50f-81d1892ab0fd'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:57.437414 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-2eee27a8-a1ae-4eed-b50f-81d1892ab0fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '835dcbfb58e54ffab5b572f3b3abc245', 'request_id': 'req-2eee27a8-a1ae-4eed-b50f-81d1892ab0fd'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:57.448384 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-2eee27a8-a1ae-4eed-b50f-81d1892ab0fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '835dcbfb58e54ffab5b572f3b3abc245', 'request_id': 'req-2eee27a8-a1ae-4eed-b50f-81d1892ab0fd'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:57.460030 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-2eee27a8-a1ae-4eed-b50f-81d1892ab0fd admin admin] The token cache is being invalidate because project 835dcbfb58e54ffab5b572f3b3abc245 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:50:57.460358 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-2eee27a8-a1ae-4eed-b50f-81d1892ab0fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:57.461474 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-2eee27a8-a1ae-4eed-b50f-81d1892ab0fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:57.463286 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 895/1807] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:50:57 2022] DELETE /identity/v3/projects/835dcbfb58e54ffab5b572f3b3abc245 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:57.467380 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6cd87b-5486-4e02-8406-636c4ee753f7 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:57.468762 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-ce6cd87b-5486-4e02-8406-636c4ee753f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:57.487064 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6cd87b-5486-4e02-8406-636c4ee753f7 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:57.565274 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f8c85c-683e-4db1-b612-c450d56a4465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:57.566641 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f8c85c-683e-4db1-b612-c450d56a4465 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:57.566937 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f8c85c-683e-4db1-b612-c450d56a4465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:57.567294 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f8c85c-683e-4db1-b612-c450d56a4465 admin admin] PATH_INFO: `/v3/users/2452d53ecab0465883487ac81183a31e` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:57.568584 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f8c85c-683e-4db1-b612-c450d56a4465 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2452d53ecab0465883487ac81183a31e)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:57.570545 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f8c85c-683e-4db1-b612-c450d56a4465 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:57.900769 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:57.919526 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-e4f8c85c-683e-4db1-b612-c450d56a4465 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2452d53ecab0465883487ac81183a31e'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:57.920225 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-e4f8c85c-683e-4db1-b612-c450d56a4465 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2452d53ecab0465883487ac81183a31e'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:57.921385 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-e4f8c85c-683e-4db1-b612-c450d56a4465 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2452d53ecab0465883487ac81183a31e'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:58.018617 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-e4f8c85c-683e-4db1-b612-c450d56a4465 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2452d53ecab0465883487ac81183a31e'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:58.038064 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-e4f8c85c-683e-4db1-b612-c450d56a4465 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2452d53ecab0465883487ac81183a31e'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:58.060280 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 913/1808] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:50:57 2022] DELETE /identity/v3/users/2452d53ecab0465883487ac81183a31e => generated 0 bytes in 593 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:58.611784 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eee27a8-a1ae-4eed-b50f-81d1892ab0fd admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:58.613612 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-2eee27a8-a1ae-4eed-b50f-81d1892ab0fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:58.639738 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eee27a8-a1ae-4eed-b50f-81d1892ab0fd admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:58.719016 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f751e8a-7ef3-45a3-bb82-55f76a6f7b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:58.719908 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f751e8a-7ef3-45a3-bb82-55f76a6f7b9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:58.720211 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f751e8a-7ef3-45a3-bb82-55f76a6f7b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:58.720491 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f751e8a-7ef3-45a3-bb82-55f76a6f7b9d admin admin] PATH_INFO: `/v3/projects/8c2abc2fdc4b43fbb3e54bf9ced6e3d8` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:58.728745 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f751e8a-7ef3-45a3-bb82-55f76a6f7b9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c2abc2fdc4b43fbb3e54bf9ced6e3d8)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:58.730694 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f751e8a-7ef3-45a3-bb82-55f76a6f7b9d admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:58.897601 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-5f751e8a-7ef3-45a3-bb82-55f76a6f7b9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c2abc2fdc4b43fbb3e54bf9ced6e3d8', 'request_id': 'req-5f751e8a-7ef3-45a3-bb82-55f76a6f7b9d'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:59.003167 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-5f751e8a-7ef3-45a3-bb82-55f76a6f7b9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c2abc2fdc4b43fbb3e54bf9ced6e3d8', 'request_id': 'req-5f751e8a-7ef3-45a3-bb82-55f76a6f7b9d'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:59.015280 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-5f751e8a-7ef3-45a3-bb82-55f76a6f7b9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c2abc2fdc4b43fbb3e54bf9ced6e3d8', 'request_id': 'req-5f751e8a-7ef3-45a3-bb82-55f76a6f7b9d'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:59.036828 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-5f751e8a-7ef3-45a3-bb82-55f76a6f7b9d admin admin] The token cache is being invalidate because project 8c2abc2fdc4b43fbb3e54bf9ced6e3d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:50:59.039723 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-5f751e8a-7ef3-45a3-bb82-55f76a6f7b9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:59.039723 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-5f751e8a-7ef3-45a3-bb82-55f76a6f7b9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:59.048266 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 896/1809] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:50:58 2022] DELETE /identity/v3/projects/8c2abc2fdc4b43fbb3e54bf9ced6e3d8 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:59.049475 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2d6396-2d66-45a0-8287-6df86c34962a None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:59.049933 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2d6396-2d66-45a0-8287-6df86c34962a None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:59.050211 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2d6396-2d66-45a0-8287-6df86c34962a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:59.094072 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-1e2d6396-2d66-45a0-8287-6df86c34962a None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:59.095680 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-1e2d6396-2d66-45a0-8287-6df86c34962a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:59.308384 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 914/1810] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:50:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:59.321673 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72edd14a-ab6e-4c4e-b02f-2eeed75a1d7c None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:59.322438 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72edd14a-ab6e-4c4e-b02f-2eeed75a1d7c None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:59.322920 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72edd14a-ab6e-4c4e-b02f-2eeed75a1d7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:59.354045 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-72edd14a-ab6e-4c4e-b02f-2eeed75a1d7c None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:59.355885 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-72edd14a-ab6e-4c4e-b02f-2eeed75a1d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:59.381484 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 897/1811] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:50:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:59.381760 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2d6396-2d66-45a0-8287-6df86c34962a None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:59.403509 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2d6396-2d66-45a0-8287-6df86c34962a None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:59.423333 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d247e87-0b64-4858-ac6d-b27e87d9a93c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:59.424370 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d247e87-0b64-4858-ac6d-b27e87d9a93c admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:59.424878 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d247e87-0b64-4858-ac6d-b27e87d9a93c admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:59.425379 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d247e87-0b64-4858-ac6d-b27e87d9a93c admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:59.426371 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d247e87-0b64-4858-ac6d-b27e87d9a93c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:50:59.426924 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d247e87-0b64-4858-ac6d-b27e87d9a93c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:59.429258 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d247e87-0b64-4858-ac6d-b27e87d9a93c admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:59.459029 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 915/1812] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jun 15 14:50:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:50:59.463704 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72edd14a-ab6e-4c4e-b02f-2eeed75a1d7c None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:59.514752 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72edd14a-ab6e-4c4e-b02f-2eeed75a1d7c None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:59.527177 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b371422-961e-4974-a633-0fc41ac0f1c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:59.527760 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b371422-961e-4974-a633-0fc41ac0f1c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:59.528071 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b371422-961e-4974-a633-0fc41ac0f1c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:59.528396 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b371422-961e-4974-a633-0fc41ac0f1c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:59.533267 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b371422-961e-4974-a633-0fc41ac0f1c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:59.535681 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b371422-961e-4974-a633-0fc41ac0f1c8 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:59.666739 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 898/1813] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:50:59 2022] POST /identity/v3/projects => generated 367 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:50:59.671205 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d247e87-0b64-4858-ac6d-b27e87d9a93c admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:59.689753 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d247e87-0b64-4858-ac6d-b27e87d9a93c admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:59.817506 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003bac6d-7047-4d4a-8919-b79716792987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:59.818276 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003bac6d-7047-4d4a-8919-b79716792987 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:59.818688 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003bac6d-7047-4d4a-8919-b79716792987 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:59.819154 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003bac6d-7047-4d4a-8919-b79716792987 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:59.821840 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003bac6d-7047-4d4a-8919-b79716792987 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:59.824061 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003bac6d-7047-4d4a-8919-b79716792987 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:59.951948 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 916/1814] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:50:59 2022] POST /identity/v3/users => generated 284 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:50:59.953388 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b371422-961e-4974-a633-0fc41ac0f1c8 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:59.960238 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:59.967816 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b371422-961e-4974-a633-0fc41ac0f1c8 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:59.981529 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf9aee5-51df-42bb-b711-e485c8ec1a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:59.982227 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf9aee5-51df-42bb-b711-e485c8ec1a75 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:59.982536 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf9aee5-51df-42bb-b711-e485c8ec1a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:59.982830 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf9aee5-51df-42bb-b711-e485c8ec1a75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:59.983604 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf9aee5-51df-42bb-b711-e485c8ec1a75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:59.985551 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf9aee5-51df-42bb-b711-e485c8ec1a75 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:00.016590 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003bac6d-7047-4d4a-8919-b79716792987 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:00.023566 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 899/1815] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:50:59 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:51:00.029705 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003bac6d-7047-4d4a-8919-b79716792987 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:00.046779 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabe7de2-b359-474d-8d55-d8aa1d1f58c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:00.047409 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabe7de2-b359-474d-8d55-d8aa1d1f58c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:00.047698 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabe7de2-b359-474d-8d55-d8aa1d1f58c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:00.047976 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabe7de2-b359-474d-8d55-d8aa1d1f58c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:00.048763 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabe7de2-b359-474d-8d55-d8aa1d1f58c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:00.050372 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabe7de2-b359-474d-8d55-d8aa1d1f58c9 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:00.062830 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 917/1816] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:51:00 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:51:00.067377 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf9aee5-51df-42bb-b711-e485c8ec1a75 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:00.079821 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf9aee5-51df-42bb-b711-e485c8ec1a75 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:00.090548 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2224c1b3-8f97-4913-becd-24eeb465bc93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:00.091547 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2224c1b3-8f97-4913-becd-24eeb465bc93 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:00.091839 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2224c1b3-8f97-4913-becd-24eeb465bc93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:00.092341 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2224c1b3-8f97-4913-becd-24eeb465bc93 admin admin] PATH_INFO: `/v3/projects/524b80e317bd4cb1a8f7002f82ba8693/users/162a6e0a087749b1a523a761c686176e/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:00.125024 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2224c1b3-8f97-4913-becd-24eeb465bc93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=524b80e317bd4cb1a8f7002f82ba8693, user_id=162a6e0a087749b1a523a761c686176e, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:00.145327 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2224c1b3-8f97-4913-becd-24eeb465bc93 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:00.281131 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 900/1817] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:51:00 2022] PUT /identity/v3/projects/524b80e317bd4cb1a8f7002f82ba8693/users/162a6e0a087749b1a523a761c686176e/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:51:00.292881 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c677cc14-e88b-46b4-bec8-e279e87c20a3 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:00.293379 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c677cc14-e88b-46b4-bec8-e279e87c20a3 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:00.293813 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c677cc14-e88b-46b4-bec8-e279e87c20a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:00.364294 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-c677cc14-e88b-46b4-bec8-e279e87c20a3 None None] MFA Rules not processed for user `162a6e0a087749b1a523a761c686176e`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:00.365602 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c677cc14-e88b-46b4-bec8-e279e87c20a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:00.384818 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:00.530600 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 918/1818] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:51:00 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:00.534011 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2224c1b3-8f97-4913-becd-24eeb465bc93 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:00.635872 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2224c1b3-8f97-4913-becd-24eeb465bc93 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:00.644081 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045ca7b0-0eeb-43ce-96cc-24574703cd9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:00.644747 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045ca7b0-0eeb-43ce-96cc-24574703cd9c admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:00.645200 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045ca7b0-0eeb-43ce-96cc-24574703cd9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:00.645650 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045ca7b0-0eeb-43ce-96cc-24574703cd9c admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:00.646818 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045ca7b0-0eeb-43ce-96cc-24574703cd9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:00.649302 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045ca7b0-0eeb-43ce-96cc-24574703cd9c admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:00.765176 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 901/1819] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:51:00 2022] POST /identity/v3/projects => generated 365 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:51:00.772658 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c677cc14-e88b-46b4-bec8-e279e87c20a3 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:00.795063 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c677cc14-e88b-46b4-bec8-e279e87c20a3 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:00.873676 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045ca7b0-0eeb-43ce-96cc-24574703cd9c admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:00.876033 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-045ca7b0-0eeb-43ce-96cc-24574703cd9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:00.933749 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b864d19-3161-4820-9d7b-b0890c395b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:00.934669 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b864d19-3161-4820-9d7b-b0890c395b4c admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:00.935078 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b864d19-3161-4820-9d7b-b0890c395b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:00.935548 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b864d19-3161-4820-9d7b-b0890c395b4c admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:00.936586 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b864d19-3161-4820-9d7b-b0890c395b4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:00.939576 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b864d19-3161-4820-9d7b-b0890c395b4c admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:01.088796 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 919/1820] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:51:00 2022] POST /identity/v3/users => generated 289 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:51:01.095288 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5415b167-94f2-400c-9463-9532e8dc949f None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:01.099040 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5415b167-94f2-400c-9463-9532e8dc949f None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:01.099504 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5415b167-94f2-400c-9463-9532e8dc949f None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:01.103279 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 920/1821] 10.222.0.42 () {54 vars in 921 bytes} [Wed Jun 15 14:51:01 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:51:01.104988 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5415b167-94f2-400c-9463-9532e8dc949f None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:01.117100 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045ca7b0-0eeb-43ce-96cc-24574703cd9c admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:01.142859 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d55a67-f399-4f6f-8332-788c08628a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:01.144032 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d55a67-f399-4f6f-8332-788c08628a96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:01.146874 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d55a67-f399-4f6f-8332-788c08628a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:01.147287 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d55a67-f399-4f6f-8332-788c08628a96 admin admin] PATH_INFO: `/v3/users/5c4f59ffdc7249648369b34c33ecb43f` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:01.150244 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5415b167-94f2-400c-9463-9532e8dc949f None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:01.150399 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d55a67-f399-4f6f-8332-788c08628a96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c4f59ffdc7249648369b34c33ecb43f)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:01.157825 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d55a67-f399-4f6f-8332-788c08628a96 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:01.168690 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07049bc8-7021-4862-82d2-652ce414bd49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:01.169499 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07049bc8-7021-4862-82d2-652ce414bd49 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:01.169855 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07049bc8-7021-4862-82d2-652ce414bd49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:01.170269 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07049bc8-7021-4862-82d2-652ce414bd49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:01.171723 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07049bc8-7021-4862-82d2-652ce414bd49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:01.174721 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07049bc8-7021-4862-82d2-652ce414bd49 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:01.210523 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 921/1822] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:51:01 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:51:01.214433 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98576823-bb91-4122-84fa-2f988021fab2 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:01.214791 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98576823-bb91-4122-84fa-2f988021fab2 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:01.219051 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98576823-bb91-4122-84fa-2f988021fab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:01.296677 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-98576823-bb91-4122-84fa-2f988021fab2 None None] MFA Rules not processed for user `0927c05c340c48e48b18754888aa9c35`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:01.302004 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-98576823-bb91-4122-84fa-2f988021fab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:01.459763 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 922/1823] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Jun 15 14:51:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:01.462129 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98576823-bb91-4122-84fa-2f988021fab2 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:01.501123 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98576823-bb91-4122-84fa-2f988021fab2 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:01.513164 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-58d55a67-f399-4f6f-8332-788c08628a96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c4f59ffdc7249648369b34c33ecb43f'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:01.513164 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-58d55a67-f399-4f6f-8332-788c08628a96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c4f59ffdc7249648369b34c33ecb43f'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:01.541510 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae441df2-4373-45d4-8552-ebf0421e3f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:01.543526 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae441df2-4373-45d4-8552-ebf0421e3f38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:01.548917 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae441df2-4373-45d4-8552-ebf0421e3f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:01.548917 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae441df2-4373-45d4-8552-ebf0421e3f38 admin admin] PATH_INFO: `/v3/projects/a5d06816bda04b9fa404e94c0526def0/users/d2d3d4cf21934777b765aa25109ca858/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:01.550776 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:01.557855 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-58d55a67-f399-4f6f-8332-788c08628a96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c4f59ffdc7249648369b34c33ecb43f'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:01.560145 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-58d55a67-f399-4f6f-8332-788c08628a96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c4f59ffdc7249648369b34c33ecb43f'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:01.577944 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae441df2-4373-45d4-8552-ebf0421e3f38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5d06816bda04b9fa404e94c0526def0, user_id=d2d3d4cf21934777b765aa25109ca858, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:01.588382 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae441df2-4373-45d4-8552-ebf0421e3f38 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:01.594416 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:01.726829 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-58d55a67-f399-4f6f-8332-788c08628a96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c4f59ffdc7249648369b34c33ecb43f'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:01.779246 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 923/1824] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:51:01 2022] PUT /identity/v3/projects/a5d06816bda04b9fa404e94c0526def0/users/d2d3d4cf21934777b765aa25109ca858/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:01.786705 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae441df2-4373-45d4-8552-ebf0421e3f38 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:01.794204 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-ae441df2-4373-45d4-8552-ebf0421e3f38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:01.822764 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 902/1825] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:51:00 2022] DELETE /identity/v3/users/5c4f59ffdc7249648369b34c33ecb43f => generated 0 bytes in 949 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:01.823708 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d55a67-f399-4f6f-8332-788c08628a96 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:01.827551 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-58d55a67-f399-4f6f-8332-788c08628a96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:01.971348 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae441df2-4373-45d4-8552-ebf0421e3f38 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:01.997418 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d55a67-f399-4f6f-8332-788c08628a96 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:02.008592 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c06040-8439-4557-9ecc-3ff50acdf6ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:02.010786 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c06040-8439-4557-9ecc-3ff50acdf6ba admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:02.010786 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c06040-8439-4557-9ecc-3ff50acdf6ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:02.010786 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c06040-8439-4557-9ecc-3ff50acdf6ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:02.010786 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c06040-8439-4557-9ecc-3ff50acdf6ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:02.016214 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c06040-8439-4557-9ecc-3ff50acdf6ba admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:02.042391 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 903/1826] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:51:01 2022] GET /identity/v3/roles => generated 1420 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:51:02.047127 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66586da-bf89-4b4d-a130-e41c159878af None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:02.051490 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66586da-bf89-4b4d-a130-e41c159878af None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:02.051941 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66586da-bf89-4b4d-a130-e41c159878af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:02.097478 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-e66586da-bf89-4b4d-a130-e41c159878af None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:02.105024 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-e66586da-bf89-4b4d-a130-e41c159878af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:02.124363 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d905beb-724b-4027-9188-a3155df3cdb6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0927c05c340c48e48b18754888aa9c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:02.125193 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d905beb-724b-4027-9188-a3155df3cdb6 service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:02.125581 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d905beb-724b-4027-9188-a3155df3cdb6 service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:02.125957 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d905beb-724b-4027-9188-a3155df3cdb6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:02.130136 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-3d905beb-724b-4027-9188-a3155df3cdb6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:02.132224 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 904/1827] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:51:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:02.137233 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66586da-bf89-4b4d-a130-e41c159878af None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:02.162773 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d905beb-724b-4027-9188-a3155df3cdb6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:02.167703 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66586da-bf89-4b4d-a130-e41c159878af None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:02.168151 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d905beb-724b-4027-9188-a3155df3cdb6 service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:02.180080 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a162d2-1171-472f-912e-10e9852e6939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:02.181596 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a162d2-1171-472f-912e-10e9852e6939 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:02.181793 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a162d2-1171-472f-912e-10e9852e6939 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:02.182241 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a162d2-1171-472f-912e-10e9852e6939 admin admin] PATH_INFO: `/v3/projects/a5d06816bda04b9fa404e94c0526def0/users/d2d3d4cf21934777b765aa25109ca858/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:02.184581 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a162d2-1171-472f-912e-10e9852e6939 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5d06816bda04b9fa404e94c0526def0, user_id=d2d3d4cf21934777b765aa25109ca858, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:02.186663 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 924/1828] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:51:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:51:02.187486 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a162d2-1171-472f-912e-10e9852e6939 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:02.187641 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d905beb-724b-4027-9188-a3155df3cdb6 service nova] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:02.189797 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-3d905beb-724b-4027-9188-a3155df3cdb6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:02.212525 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 905/1829] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:51:02 2022] PUT /identity/v3/projects/a5d06816bda04b9fa404e94c0526def0/users/d2d3d4cf21934777b765aa25109ca858/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:02.224774 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa0f16-a6a6-4292-8cea-2171bcb6b9ae None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:02.225130 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa0f16-a6a6-4292-8cea-2171bcb6b9ae None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:02.225502 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa0f16-a6a6-4292-8cea-2171bcb6b9ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:02.277670 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-a4fa0f16-a6a6-4292-8cea-2171bcb6b9ae None None] MFA Rules not processed for user `d2d3d4cf21934777b765aa25109ca858`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:02.279915 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-a4fa0f16-a6a6-4292-8cea-2171bcb6b9ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:02.298162 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d905beb-724b-4027-9188-a3155df3cdb6 service nova] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:02.365436 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:02.397063 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea333b6-67c9-4d16-b5ba-4916ee9a13a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:02.397301 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea333b6-67c9-4d16-b5ba-4916ee9a13a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:02.397768 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea333b6-67c9-4d16-b5ba-4916ee9a13a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:02.398409 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea333b6-67c9-4d16-b5ba-4916ee9a13a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:02.474737 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 906/1830] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:51:02 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:02.500305 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea333b6-67c9-4d16-b5ba-4916ee9a13a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:02.501887 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea333b6-67c9-4d16-b5ba-4916ee9a13a1 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:02.514044 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 925/1831] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:51:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:51:02.620413 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fa0f16-a6a6-4292-8cea-2171bcb6b9ae None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:02.622027 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-a4fa0f16-a6a6-4292-8cea-2171bcb6b9ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:02.659651 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:02.685580 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fa0f16-a6a6-4292-8cea-2171bcb6b9ae None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:02.689391 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90433380-a7a7-47ac-a572-1159051633cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:02.690045 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90433380-a7a7-47ac-a572-1159051633cd service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:02.690331 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90433380-a7a7-47ac-a572-1159051633cd service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:02.690611 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90433380-a7a7-47ac-a572-1159051633cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:02.700799 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90433380-a7a7-47ac-a572-1159051633cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:02.702330 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90433380-a7a7-47ac-a572-1159051633cd service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:02.712544 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 907/1832] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:51:02 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:51:03.033056 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea333b6-67c9-4d16-b5ba-4916ee9a13a1 service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:03.051811 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea333b6-67c9-4d16-b5ba-4916ee9a13a1 service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:03.051811 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c82761-4745-4804-840e-67f78f5ccbb9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:03.051811 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c82761-4745-4804-840e-67f78f5ccbb9 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:03.051811 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c82761-4745-4804-840e-67f78f5ccbb9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:03.052173 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c82761-4745-4804-840e-67f78f5ccbb9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:03.053609 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e5c82761-4745-4804-840e-67f78f5ccbb9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:03.054760 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90433380-a7a7-47ac-a572-1159051633cd service cinder] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:03.057031 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-90433380-a7a7-47ac-a572-1159051633cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:03.071880 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90433380-a7a7-47ac-a572-1159051633cd service cinder] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:03.077328 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9375f75-69f0-4331-9719-c4084b592163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:03.079503 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9375f75-69f0-4331-9719-c4084b592163 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:03.080307 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9375f75-69f0-4331-9719-c4084b592163 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:03.081029 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9375f75-69f0-4331-9719-c4084b592163 admin admin] PATH_INFO: `/v3/projects/45d897d3d61c4767959f1f6a33ba2694` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:03.086226 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9375f75-69f0-4331-9719-c4084b592163 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45d897d3d61c4767959f1f6a33ba2694)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:03.086226 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9375f75-69f0-4331-9719-c4084b592163 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:03.134921 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c82761-4745-4804-840e-67f78f5ccbb9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:03.136693 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c82761-4745-4804-840e-67f78f5ccbb9 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:03.147571 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 926/1833] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:51:03 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:51:03.222008 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-d9375f75-69f0-4331-9719-c4084b592163 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45d897d3d61c4767959f1f6a33ba2694', 'request_id': 'req-d9375f75-69f0-4331-9719-c4084b592163'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:03.317596 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-d9375f75-69f0-4331-9719-c4084b592163 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45d897d3d61c4767959f1f6a33ba2694', 'request_id': 'req-d9375f75-69f0-4331-9719-c4084b592163'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:03.325406 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-d9375f75-69f0-4331-9719-c4084b592163 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45d897d3d61c4767959f1f6a33ba2694', 'request_id': 'req-d9375f75-69f0-4331-9719-c4084b592163'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:03.338807 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-d9375f75-69f0-4331-9719-c4084b592163 admin admin] The token cache is being invalidate because project 45d897d3d61c4767959f1f6a33ba2694 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:51:03.339088 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-d9375f75-69f0-4331-9719-c4084b592163 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:03.339759 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-d9375f75-69f0-4331-9719-c4084b592163 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:03.341404 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 908/1834] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:51:03 2022] DELETE /identity/v3/projects/45d897d3d61c4767959f1f6a33ba2694 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:51:03.345517 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c82761-4745-4804-840e-67f78f5ccbb9 service cinder] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:03.362063 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c82761-4745-4804-840e-67f78f5ccbb9 service cinder] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:03.436942 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab5182e-3a9c-40af-867a-7ab91491511c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:03.437925 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab5182e-3a9c-40af-867a-7ab91491511c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:03.438309 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab5182e-3a9c-40af-867a-7ab91491511c admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:03.438724 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab5182e-3a9c-40af-867a-7ab91491511c admin admin] PATH_INFO: `/v3/users/c69a17a612c644c0b4b4cdc7f7268059` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:03.440203 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab5182e-3a9c-40af-867a-7ab91491511c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c69a17a612c644c0b4b4cdc7f7268059)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:03.446450 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab5182e-3a9c-40af-867a-7ab91491511c admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:03.613962 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:03.703660 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5ab5182e-3a9c-40af-867a-7ab91491511c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c69a17a612c644c0b4b4cdc7f7268059'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:03.704768 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5ab5182e-3a9c-40af-867a-7ab91491511c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c69a17a612c644c0b4b4cdc7f7268059'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:03.705672 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5ab5182e-3a9c-40af-867a-7ab91491511c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c69a17a612c644c0b4b4cdc7f7268059'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:03.837048 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5ab5182e-3a9c-40af-867a-7ab91491511c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c69a17a612c644c0b4b4cdc7f7268059'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:03.856410 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5ab5182e-3a9c-40af-867a-7ab91491511c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c69a17a612c644c0b4b4cdc7f7268059'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:03.876925 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 927/1835] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:51:03 2022] DELETE /identity/v3/users/c69a17a612c644c0b4b4cdc7f7268059 => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:04.422094 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9375f75-69f0-4331-9719-c4084b592163 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:04.423670 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-d9375f75-69f0-4331-9719-c4084b592163 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:04.437874 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9375f75-69f0-4331-9719-c4084b592163 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:04.501151 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1fbd4f-e1f0-4518-b0bd-11823629b70e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:04.502045 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1fbd4f-e1f0-4518-b0bd-11823629b70e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:04.502323 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1fbd4f-e1f0-4518-b0bd-11823629b70e admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:04.502595 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1fbd4f-e1f0-4518-b0bd-11823629b70e admin admin] PATH_INFO: `/v3/projects/315b456cd6f44cebaf0a9a920bb5da44` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:04.503783 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1fbd4f-e1f0-4518-b0bd-11823629b70e admin admin] RBAC: Authorizing `identity:delete_project(project_id=315b456cd6f44cebaf0a9a920bb5da44)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:04.505777 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1fbd4f-e1f0-4518-b0bd-11823629b70e admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:04.645740 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-8e1fbd4f-e1f0-4518-b0bd-11823629b70e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '315b456cd6f44cebaf0a9a920bb5da44', 'request_id': 'req-8e1fbd4f-e1f0-4518-b0bd-11823629b70e'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:04.740480 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-8e1fbd4f-e1f0-4518-b0bd-11823629b70e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '315b456cd6f44cebaf0a9a920bb5da44', 'request_id': 'req-8e1fbd4f-e1f0-4518-b0bd-11823629b70e'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:04.747580 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:04.750276 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-8e1fbd4f-e1f0-4518-b0bd-11823629b70e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '315b456cd6f44cebaf0a9a920bb5da44', 'request_id': 'req-8e1fbd4f-e1f0-4518-b0bd-11823629b70e'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:04.762573 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-8e1fbd4f-e1f0-4518-b0bd-11823629b70e admin admin] The token cache is being invalidate because project 315b456cd6f44cebaf0a9a920bb5da44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:51:04.762882 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-8e1fbd4f-e1f0-4518-b0bd-11823629b70e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:04.764033 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-8e1fbd4f-e1f0-4518-b0bd-11823629b70e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:04.765238 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 909/1836] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:51:04 2022] DELETE /identity/v3/projects/315b456cd6f44cebaf0a9a920bb5da44 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:04.773016 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b248481-b0c1-4f9e-a9ce-9340be37ac35 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:04.773599 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b248481-b0c1-4f9e-a9ce-9340be37ac35 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:04.773762 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b248481-b0c1-4f9e-a9ce-9340be37ac35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:04.792872 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-4b248481-b0c1-4f9e-a9ce-9340be37ac35 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:04.794051 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-4b248481-b0c1-4f9e-a9ce-9340be37ac35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:04.870386 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 928/1837] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:51:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:51:04.872817 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a8341-793c-495c-8d5a-68319bd5bb1e None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:04.873224 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a8341-793c-495c-8d5a-68319bd5bb1e None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:04.873591 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a8341-793c-495c-8d5a-68319bd5bb1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:04.894305 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-5f1a8341-793c-495c-8d5a-68319bd5bb1e None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:04.895552 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-5f1a8341-793c-495c-8d5a-68319bd5bb1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:04.905287 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 910/1838] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:51:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:04.910034 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b248481-b0c1-4f9e-a9ce-9340be37ac35 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:04.923228 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b248481-b0c1-4f9e-a9ce-9340be37ac35 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:04.928222 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0a1e08-39d7-4a20-a507-2ee70293a1ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:04.928897 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0a1e08-39d7-4a20-a507-2ee70293a1ee admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:04.929182 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0a1e08-39d7-4a20-a507-2ee70293a1ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:04.929502 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0a1e08-39d7-4a20-a507-2ee70293a1ee admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:04.930729 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0a1e08-39d7-4a20-a507-2ee70293a1ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:51:04.931039 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0a1e08-39d7-4a20-a507-2ee70293a1ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:04.932683 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0a1e08-39d7-4a20-a507-2ee70293a1ee admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:04.948656 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 929/1839] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jun 15 14:51:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:51:04.949946 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1a8341-793c-495c-8d5a-68319bd5bb1e None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:04.959283 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1a8341-793c-495c-8d5a-68319bd5bb1e None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:04.964942 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f7e744-3a8a-467c-9069-a8a9e37aaa60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:04.965603 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7e744-3a8a-467c-9069-a8a9e37aaa60 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:04.965890 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7e744-3a8a-467c-9069-a8a9e37aaa60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:04.966228 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7e744-3a8a-467c-9069-a8a9e37aaa60 admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:04.966932 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f7e744-3a8a-467c-9069-a8a9e37aaa60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:04.968921 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f7e744-3a8a-467c-9069-a8a9e37aaa60 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:05.054867 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 911/1840] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:51:04 2022] POST /identity/v3/projects => generated 365 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:51:05.058959 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0a1e08-39d7-4a20-a507-2ee70293a1ee admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:05.069076 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0a1e08-39d7-4a20-a507-2ee70293a1ee admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:05.133412 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80603b35-f9c2-4dcf-96fe-836ee26f71f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:05.134100 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80603b35-f9c2-4dcf-96fe-836ee26f71f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:05.134403 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80603b35-f9c2-4dcf-96fe-836ee26f71f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:05.134663 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80603b35-f9c2-4dcf-96fe-836ee26f71f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:05.135359 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80603b35-f9c2-4dcf-96fe-836ee26f71f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:05.137724 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80603b35-f9c2-4dcf-96fe-836ee26f71f1 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:05.266003 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 930/1841] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:51:05 2022] POST /identity/v3/users => generated 283 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:51:05.270273 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f7e744-3a8a-467c-9069-a8a9e37aaa60 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:05.284412 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f7e744-3a8a-467c-9069-a8a9e37aaa60 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:05.290518 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f862e3a7-a695-488e-a53e-dad697565646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:05.291457 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f862e3a7-a695-488e-a53e-dad697565646 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:05.291876 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f862e3a7-a695-488e-a53e-dad697565646 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:05.292401 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f862e3a7-a695-488e-a53e-dad697565646 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:05.293140 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f862e3a7-a695-488e-a53e-dad697565646 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:05.294750 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f862e3a7-a695-488e-a53e-dad697565646 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:05.310025 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 912/1842] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:51:05 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:51:05.314457 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80603b35-f9c2-4dcf-96fe-836ee26f71f1 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:05.325269 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80603b35-f9c2-4dcf-96fe-836ee26f71f1 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:05.330599 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1016787f-2ade-43f0-9782-e03f1261dfcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:05.331267 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1016787f-2ade-43f0-9782-e03f1261dfcf admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:05.331562 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1016787f-2ade-43f0-9782-e03f1261dfcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:05.331839 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1016787f-2ade-43f0-9782-e03f1261dfcf admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:05.332654 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1016787f-2ade-43f0-9782-e03f1261dfcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:05.334244 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1016787f-2ade-43f0-9782-e03f1261dfcf admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:05.356234 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 931/1843] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:51:05 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:51:05.357620 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f862e3a7-a695-488e-a53e-dad697565646 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:05.367936 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f862e3a7-a695-488e-a53e-dad697565646 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:05.373861 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd7d03c-d60d-48bf-b9b3-3284816c9887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:05.374790 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd7d03c-d60d-48bf-b9b3-3284816c9887 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:05.375074 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd7d03c-d60d-48bf-b9b3-3284816c9887 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:05.375352 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd7d03c-d60d-48bf-b9b3-3284816c9887 admin admin] PATH_INFO: `/v3/projects/1fed6c0fa85343619e76cba419f05300/users/6cd68a9d4b2649f8a263bc083cfa6cb0/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:05.391080 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd7d03c-d60d-48bf-b9b3-3284816c9887 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fed6c0fa85343619e76cba419f05300, user_id=6cd68a9d4b2649f8a263bc083cfa6cb0, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:05.393840 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd7d03c-d60d-48bf-b9b3-3284816c9887 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:05.500788 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 913/1844] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:51:05 2022] PUT /identity/v3/projects/1fed6c0fa85343619e76cba419f05300/users/6cd68a9d4b2649f8a263bc083cfa6cb0/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:05.518987 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606b7e64-a152-4442-9b4d-f7d4eec83943 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:05.519308 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606b7e64-a152-4442-9b4d-f7d4eec83943 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:05.519588 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606b7e64-a152-4442-9b4d-f7d4eec83943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:05.544205 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:05.554473 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-606b7e64-a152-4442-9b4d-f7d4eec83943 None None] MFA Rules not processed for user `6cd68a9d4b2649f8a263bc083cfa6cb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:05.556583 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-606b7e64-a152-4442-9b4d-f7d4eec83943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:05.628314 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 932/1845] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:51:05 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:05.632999 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd7d03c-d60d-48bf-b9b3-3284816c9887 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:05.709392 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd7d03c-d60d-48bf-b9b3-3284816c9887 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:05.714344 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761b9b9a-69d9-47f2-8d90-af556ee835c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:05.715031 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761b9b9a-69d9-47f2-8d90-af556ee835c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:05.715370 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761b9b9a-69d9-47f2-8d90-af556ee835c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:05.715645 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761b9b9a-69d9-47f2-8d90-af556ee835c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:05.716382 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761b9b9a-69d9-47f2-8d90-af556ee835c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:05.718267 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761b9b9a-69d9-47f2-8d90-af556ee835c2 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:05.827135 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 914/1846] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:51:05 2022] POST /identity/v3/projects => generated 363 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:51:05.831625 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606b7e64-a152-4442-9b4d-f7d4eec83943 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:05.844701 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606b7e64-a152-4442-9b4d-f7d4eec83943 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:05.903953 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f0b8f0-376b-4643-9ed8-741a92748c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:05.904811 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f0b8f0-376b-4643-9ed8-741a92748c63 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:05.905094 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f0b8f0-376b-4643-9ed8-741a92748c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:05.905364 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f0b8f0-376b-4643-9ed8-741a92748c63 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:05.906054 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f0b8f0-376b-4643-9ed8-741a92748c63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:05.907970 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f0b8f0-376b-4643-9ed8-741a92748c63 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:06.033732 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 933/1847] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:51:05 2022] POST /identity/v3/users => generated 288 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:51:06.037873 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761b9b9a-69d9-47f2-8d90-af556ee835c2 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:06.050555 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761b9b9a-69d9-47f2-8d90-af556ee835c2 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:06.055773 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6588f2c-ffa2-4d5f-8801-6ffac48d4eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:06.056486 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6588f2c-ffa2-4d5f-8801-6ffac48d4eab admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:06.056846 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6588f2c-ffa2-4d5f-8801-6ffac48d4eab admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:06.057192 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6588f2c-ffa2-4d5f-8801-6ffac48d4eab admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:06.058027 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6588f2c-ffa2-4d5f-8801-6ffac48d4eab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:06.059926 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6588f2c-ffa2-4d5f-8801-6ffac48d4eab admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:06.075150 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 915/1848] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:51:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:51:06.076381 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f0b8f0-376b-4643-9ed8-741a92748c63 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:06.088559 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f0b8f0-376b-4643-9ed8-741a92748c63 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:06.094520 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6997155-2630-48b7-987a-156910cf419c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:06.095423 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6997155-2630-48b7-987a-156910cf419c admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:06.095708 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6997155-2630-48b7-987a-156910cf419c admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:06.095982 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6997155-2630-48b7-987a-156910cf419c admin admin] PATH_INFO: `/v3/projects/e79be289c83e42c7905de1e0ef72e0b5/users/0f450004a43944c9b0636fb1620ead4b/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:06.125550 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6997155-2630-48b7-987a-156910cf419c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e79be289c83e42c7905de1e0ef72e0b5, user_id=0f450004a43944c9b0636fb1620ead4b, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:06.128639 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6997155-2630-48b7-987a-156910cf419c admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:06.209266 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 934/1849] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:51:06 2022] PUT /identity/v3/projects/e79be289c83e42c7905de1e0ef72e0b5/users/0f450004a43944c9b0636fb1620ead4b/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:06.214687 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6588f2c-ffa2-4d5f-8801-6ffac48d4eab admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:06.235234 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6588f2c-ffa2-4d5f-8801-6ffac48d4eab admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:06.284598 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45053d9-02e2-460c-8d5a-59a3edb1d370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:06.285448 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45053d9-02e2-460c-8d5a-59a3edb1d370 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:06.285840 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45053d9-02e2-460c-8d5a-59a3edb1d370 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:06.286214 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45053d9-02e2-460c-8d5a-59a3edb1d370 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:06.287347 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45053d9-02e2-460c-8d5a-59a3edb1d370 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:06.289122 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45053d9-02e2-460c-8d5a-59a3edb1d370 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:06.290520 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:06.300349 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 916/1850] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:51:06 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:51:06.304632 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6997155-2630-48b7-987a-156910cf419c admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:06.315310 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6997155-2630-48b7-987a-156910cf419c admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:06.320340 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fe0728-28e7-4f9b-8270-8c46b1a1880d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:06.321349 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe0728-28e7-4f9b-8270-8c46b1a1880d admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:06.321636 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe0728-28e7-4f9b-8270-8c46b1a1880d admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:06.321913 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe0728-28e7-4f9b-8270-8c46b1a1880d admin admin] PATH_INFO: `/v3/projects/e79be289c83e42c7905de1e0ef72e0b5/users/0f450004a43944c9b0636fb1620ead4b/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:06.323958 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fe0728-28e7-4f9b-8270-8c46b1a1880d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e79be289c83e42c7905de1e0ef72e0b5, user_id=0f450004a43944c9b0636fb1620ead4b, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:06.326722 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fe0728-28e7-4f9b-8270-8c46b1a1880d admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:06.338598 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 935/1851] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:51:06 2022] PUT /identity/v3/projects/e79be289c83e42c7905de1e0ef72e0b5/users/0f450004a43944c9b0636fb1620ead4b/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:06.349002 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406fee9e-1319-4032-916d-c76a4bb80065 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:06.349332 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406fee9e-1319-4032-916d-c76a4bb80065 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:06.349648 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406fee9e-1319-4032-916d-c76a4bb80065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:06.378930 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-406fee9e-1319-4032-916d-c76a4bb80065 None None] MFA Rules not processed for user `0f450004a43944c9b0636fb1620ead4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:06.380160 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-406fee9e-1319-4032-916d-c76a4bb80065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:06.455972 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 917/1852] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:51:06 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:06.468651 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fe0728-28e7-4f9b-8270-8c46b1a1880d admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:06.470636 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-12fe0728-28e7-4f9b-8270-8c46b1a1880d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:06.535409 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fe0728-28e7-4f9b-8270-8c46b1a1880d admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:06.542933 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf34920d-702c-47a2-9943-8e905e68cca9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:06.543535 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf34920d-702c-47a2-9943-8e905e68cca9 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:06.543816 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf34920d-702c-47a2-9943-8e905e68cca9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:06.544190 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf34920d-702c-47a2-9943-8e905e68cca9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:06.553783 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:06.558363 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf34920d-702c-47a2-9943-8e905e68cca9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:06.559812 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf34920d-702c-47a2-9943-8e905e68cca9 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:06.571125 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 936/1853] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:51:06 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:51:06.794895 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406fee9e-1319-4032-916d-c76a4bb80065 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:06.805654 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406fee9e-1319-4032-916d-c76a4bb80065 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:06.809633 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e25b79-e8c9-466b-943e-3be3bb33ae20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:06.810304 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e25b79-e8c9-466b-943e-3be3bb33ae20 service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:06.810595 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e25b79-e8c9-466b-943e-3be3bb33ae20 service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:06.810872 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e25b79-e8c9-466b-943e-3be3bb33ae20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:06.872202 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e25b79-e8c9-466b-943e-3be3bb33ae20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:06.873612 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e25b79-e8c9-466b-943e-3be3bb33ae20 service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:06.884323 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 918/1854] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:51:06 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:51:11.113785 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf34920d-702c-47a2-9943-8e905e68cca9 service cinder] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:11.201134 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf34920d-702c-47a2-9943-8e905e68cca9 service cinder] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:11.207586 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b86774-4a3d-477d-a64f-5c749be51e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:11.208471 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b86774-4a3d-477d-a64f-5c749be51e36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:11.208753 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b86774-4a3d-477d-a64f-5c749be51e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:11.209033 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b86774-4a3d-477d-a64f-5c749be51e36 admin admin] PATH_INFO: `/v3/users/6cd68a9d4b2649f8a263bc083cfa6cb0` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:11.211541 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b86774-4a3d-477d-a64f-5c749be51e36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6cd68a9d4b2649f8a263bc083cfa6cb0)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:11.213526 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b86774-4a3d-477d-a64f-5c749be51e36 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:11.450482 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-97b86774-4a3d-477d-a64f-5c749be51e36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6cd68a9d4b2649f8a263bc083cfa6cb0'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:11.451246 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-97b86774-4a3d-477d-a64f-5c749be51e36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6cd68a9d4b2649f8a263bc083cfa6cb0'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:11.451961 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-97b86774-4a3d-477d-a64f-5c749be51e36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6cd68a9d4b2649f8a263bc083cfa6cb0'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:11.529318 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-97b86774-4a3d-477d-a64f-5c749be51e36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6cd68a9d4b2649f8a263bc083cfa6cb0'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:11.578826 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-97b86774-4a3d-477d-a64f-5c749be51e36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6cd68a9d4b2649f8a263bc083cfa6cb0'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:11.611675 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 937/1855] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:51:11 2022] DELETE /identity/v3/users/6cd68a9d4b2649f8a263bc083cfa6cb0 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:11.615887 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d8c2f6-1c8b-44c8-b0b8-7f002c933ab6 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:11.618700 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d8c2f6-1c8b-44c8-b0b8-7f002c933ab6 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:11.619147 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d8c2f6-1c8b-44c8-b0b8-7f002c933ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:11.667594 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-83d8c2f6-1c8b-44c8-b0b8-7f002c933ab6 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:11.669528 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-83d8c2f6-1c8b-44c8-b0b8-7f002c933ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:11.748987 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 919/1856] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:51:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:11.760472 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b86774-4a3d-477d-a64f-5c749be51e36 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:11.762215 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-97b86774-4a3d-477d-a64f-5c749be51e36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:11.844960 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b86774-4a3d-477d-a64f-5c749be51e36 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:11.855561 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c9872d-fce2-4a88-85c6-cb2766631c10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:11.856292 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c9872d-fce2-4a88-85c6-cb2766631c10 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:11.856573 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c9872d-fce2-4a88-85c6-cb2766631c10 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:11.856848 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c9872d-fce2-4a88-85c6-cb2766631c10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:11.867922 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c9872d-fce2-4a88-85c6-cb2766631c10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:11.869251 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c9872d-fce2-4a88-85c6-cb2766631c10 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:11.880107 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 938/1857] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:51:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:51:12.777365 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d8c2f6-1c8b-44c8-b0b8-7f002c933ab6 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:12.779788 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-83d8c2f6-1c8b-44c8-b0b8-7f002c933ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:12.807807 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d8c2f6-1c8b-44c8-b0b8-7f002c933ab6 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:12.832499 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381889db-3bb2-4c89-a9dd-104c9053bd34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:12.832939 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381889db-3bb2-4c89-a9dd-104c9053bd34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:12.833230 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381889db-3bb2-4c89-a9dd-104c9053bd34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:12.833516 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381889db-3bb2-4c89-a9dd-104c9053bd34 admin admin] PATH_INFO: `/v3/projects/1fed6c0fa85343619e76cba419f05300` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:12.836766 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381889db-3bb2-4c89-a9dd-104c9053bd34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fed6c0fa85343619e76cba419f05300)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:12.839191 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381889db-3bb2-4c89-a9dd-104c9053bd34 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:12.867607 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:12.973763 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-381889db-3bb2-4c89-a9dd-104c9053bd34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fed6c0fa85343619e76cba419f05300', 'request_id': 'req-381889db-3bb2-4c89-a9dd-104c9053bd34'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:13.054743 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-381889db-3bb2-4c89-a9dd-104c9053bd34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fed6c0fa85343619e76cba419f05300', 'request_id': 'req-381889db-3bb2-4c89-a9dd-104c9053bd34'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:13.064363 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-381889db-3bb2-4c89-a9dd-104c9053bd34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fed6c0fa85343619e76cba419f05300', 'request_id': 'req-381889db-3bb2-4c89-a9dd-104c9053bd34'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:13.075683 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-381889db-3bb2-4c89-a9dd-104c9053bd34 admin admin] The token cache is being invalidate because project 1fed6c0fa85343619e76cba419f05300 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:51:13.076249 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-381889db-3bb2-4c89-a9dd-104c9053bd34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:13.077004 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-381889db-3bb2-4c89-a9dd-104c9053bd34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:13.078661 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 920/1858] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:51:12 2022] DELETE /identity/v3/projects/1fed6c0fa85343619e76cba419f05300 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:13.083335 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c9872d-fce2-4a88-85c6-cb2766631c10 service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:13.096630 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c9872d-fce2-4a88-85c6-cb2766631c10 service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:13.114906 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:13.193110 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4b0d28-f6a1-4dae-bc0a-43558ab2e0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:13.194028 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b0d28-f6a1-4dae-bc0a-43558ab2e0d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:13.194425 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b0d28-f6a1-4dae-bc0a-43558ab2e0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:13.195212 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b0d28-f6a1-4dae-bc0a-43558ab2e0d3 admin admin] PATH_INFO: `/v3/users/0f450004a43944c9b0636fb1620ead4b` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:13.196733 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4b0d28-f6a1-4dae-bc0a-43558ab2e0d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f450004a43944c9b0636fb1620ead4b)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:13.198928 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4b0d28-f6a1-4dae-bc0a-43558ab2e0d3 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:13.485966 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5f4b0d28-f6a1-4dae-bc0a-43558ab2e0d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f450004a43944c9b0636fb1620ead4b'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:13.486674 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5f4b0d28-f6a1-4dae-bc0a-43558ab2e0d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f450004a43944c9b0636fb1620ead4b'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:13.487394 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5f4b0d28-f6a1-4dae-bc0a-43558ab2e0d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f450004a43944c9b0636fb1620ead4b'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:13.575221 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5f4b0d28-f6a1-4dae-bc0a-43558ab2e0d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f450004a43944c9b0636fb1620ead4b'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:13.602039 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5f4b0d28-f6a1-4dae-bc0a-43558ab2e0d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f450004a43944c9b0636fb1620ead4b'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:13.618206 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 939/1859] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:51:13 2022] DELETE /identity/v3/users/0f450004a43944c9b0636fb1620ead4b => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:51:14.219919 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381889db-3bb2-4c89-a9dd-104c9053bd34 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:14.222159 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-381889db-3bb2-4c89-a9dd-104c9053bd34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:14.252562 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381889db-3bb2-4c89-a9dd-104c9053bd34 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:14.329091 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1965209-1503-478c-9fd0-891f1655c32f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:14.329985 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1965209-1503-478c-9fd0-891f1655c32f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:14.330244 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1965209-1503-478c-9fd0-891f1655c32f admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:14.330516 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1965209-1503-478c-9fd0-891f1655c32f admin admin] PATH_INFO: `/v3/projects/e79be289c83e42c7905de1e0ef72e0b5` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:14.331938 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1965209-1503-478c-9fd0-891f1655c32f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e79be289c83e42c7905de1e0ef72e0b5)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:14.333924 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1965209-1503-478c-9fd0-891f1655c32f admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:14.477500 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-f1965209-1503-478c-9fd0-891f1655c32f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e79be289c83e42c7905de1e0ef72e0b5', 'request_id': 'req-f1965209-1503-478c-9fd0-891f1655c32f'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:14.591627 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-f1965209-1503-478c-9fd0-891f1655c32f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e79be289c83e42c7905de1e0ef72e0b5', 'request_id': 'req-f1965209-1503-478c-9fd0-891f1655c32f'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:14.599822 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-f1965209-1503-478c-9fd0-891f1655c32f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e79be289c83e42c7905de1e0ef72e0b5', 'request_id': 'req-f1965209-1503-478c-9fd0-891f1655c32f'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:14.614371 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-f1965209-1503-478c-9fd0-891f1655c32f admin admin] The token cache is being invalidate because project e79be289c83e42c7905de1e0ef72e0b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:51:14.614826 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-f1965209-1503-478c-9fd0-891f1655c32f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:14.615596 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-f1965209-1503-478c-9fd0-891f1655c32f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:14.617347 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 921/1860] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:51:14 2022] DELETE /identity/v3/projects/e79be289c83e42c7905de1e0ef72e0b5 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:14.633376 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b0c55-ec07-43ce-a084-db854ed2630e None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:14.634272 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b0c55-ec07-43ce-a084-db854ed2630e None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:14.634799 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b0c55-ec07-43ce-a084-db854ed2630e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:14.655238 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-a14b0c55-ec07-43ce-a084-db854ed2630e None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:14.656527 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a14b0c55-ec07-43ce-a084-db854ed2630e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:14.666555 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1965209-1503-478c-9fd0-891f1655c32f admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:14.669454 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-f1965209-1503-478c-9fd0-891f1655c32f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:14.731667 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:14.745198 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 940/1861] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:51:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:14.753827 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57214383-1b7e-42db-9a3c-3d256cd0c984 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:14.754582 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57214383-1b7e-42db-9a3c-3d256cd0c984 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:14.755430 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57214383-1b7e-42db-9a3c-3d256cd0c984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:14.785481 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-57214383-1b7e-42db-9a3c-3d256cd0c984 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:14.786852 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-57214383-1b7e-42db-9a3c-3d256cd0c984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:14.795907 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1965209-1503-478c-9fd0-891f1655c32f admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:14.796398 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 941/1862] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:51:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:14.800674 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57214383-1b7e-42db-9a3c-3d256cd0c984 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:14.802468 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80774997-70f4-4241-a086-16b4b3064870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:14.806499 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80774997-70f4-4241-a086-16b4b3064870 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:14.806499 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80774997-70f4-4241-a086-16b4b3064870 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:14.806499 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80774997-70f4-4241-a086-16b4b3064870 admin admin] PATH_INFO: `/v3/users/beb3dcb49d6f431298ce0056a972b50f` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:14.806499 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80774997-70f4-4241-a086-16b4b3064870 admin admin] RBAC: Authorizing `identity:delete_user(user_id=beb3dcb49d6f431298ce0056a972b50f)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:14.807856 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80774997-70f4-4241-a086-16b4b3064870 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:14.812216 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57214383-1b7e-42db-9a3c-3d256cd0c984 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:14.818090 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569b12f5-8912-44f3-846c-b74777d0614f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:14.818752 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569b12f5-8912-44f3-846c-b74777d0614f admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:14.819033 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569b12f5-8912-44f3-846c-b74777d0614f admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:14.819594 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569b12f5-8912-44f3-846c-b74777d0614f admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:14.821109 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569b12f5-8912-44f3-846c-b74777d0614f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:51:14.821109 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569b12f5-8912-44f3-846c-b74777d0614f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:14.823237 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569b12f5-8912-44f3-846c-b74777d0614f admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:14.838996 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 942/1863] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jun 15 14:51:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:51:14.840289 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569b12f5-8912-44f3-846c-b74777d0614f admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:14.850726 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569b12f5-8912-44f3-846c-b74777d0614f admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:14.855678 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb9eb63-7d29-4df8-bbd3-908438a3e8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:14.856367 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb9eb63-7d29-4df8-bbd3-908438a3e8fe admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:14.856649 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb9eb63-7d29-4df8-bbd3-908438a3e8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:14.860303 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb9eb63-7d29-4df8-bbd3-908438a3e8fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:14.860303 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb9eb63-7d29-4df8-bbd3-908438a3e8fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:14.860303 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb9eb63-7d29-4df8-bbd3-908438a3e8fe admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:14.874262 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:14.986642 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 943/1864] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:51:14 2022] POST /identity/v3/projects => generated 365 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:51:14.990701 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb9eb63-7d29-4df8-bbd3-908438a3e8fe admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:15.069318 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-80774997-70f4-4241-a086-16b4b3064870 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'beb3dcb49d6f431298ce0056a972b50f'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:15.069421 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-80774997-70f4-4241-a086-16b4b3064870 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'beb3dcb49d6f431298ce0056a972b50f'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:15.078036 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-80774997-70f4-4241-a086-16b4b3064870 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'beb3dcb49d6f431298ce0056a972b50f'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:15.078842 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-80774997-70f4-4241-a086-16b4b3064870 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'beb3dcb49d6f431298ce0056a972b50f'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:15.093892 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb9eb63-7d29-4df8-bbd3-908438a3e8fe admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:15.099668 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4c7d2a-b44e-499d-9994-f03e0cd9cc6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:15.102593 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c7d2a-b44e-499d-9994-f03e0cd9cc6d admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:15.102757 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c7d2a-b44e-499d-9994-f03e0cd9cc6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:15.102905 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c7d2a-b44e-499d-9994-f03e0cd9cc6d admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:15.103647 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4c7d2a-b44e-499d-9994-f03e0cd9cc6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:15.105926 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4c7d2a-b44e-499d-9994-f03e0cd9cc6d admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:15.162568 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-80774997-70f4-4241-a086-16b4b3064870 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'beb3dcb49d6f431298ce0056a972b50f'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:15.183993 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 922/1865] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:51:14 2022] DELETE /identity/v3/users/beb3dcb49d6f431298ce0056a972b50f => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:15.192521 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87667d6-41bb-48e5-a054-b507e828e998 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:15.192907 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87667d6-41bb-48e5-a054-b507e828e998 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:15.193191 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87667d6-41bb-48e5-a054-b507e828e998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:15.213522 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-c87667d6-41bb-48e5-a054-b507e828e998 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:15.215001 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-c87667d6-41bb-48e5-a054-b507e828e998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:15.222232 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 944/1866] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:51:14 2022] POST /identity/v3/users => generated 283 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:51:15.227761 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4c7d2a-b44e-499d-9994-f03e0cd9cc6d admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:15.229511 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-7d4c7d2a-b44e-499d-9994-f03e0cd9cc6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:15.245045 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4c7d2a-b44e-499d-9994-f03e0cd9cc6d admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:15.289616 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 923/1867] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:51:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:51:15.303309 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87667d6-41bb-48e5-a054-b507e828e998 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:15.305872 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-c87667d6-41bb-48e5-a054-b507e828e998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:15.330328 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98077f1a-37d2-4c73-a955-92b88183f999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:15.331372 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98077f1a-37d2-4c73-a955-92b88183f999 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:15.331848 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98077f1a-37d2-4c73-a955-92b88183f999 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:15.332437 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98077f1a-37d2-4c73-a955-92b88183f999 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:15.333545 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98077f1a-37d2-4c73-a955-92b88183f999 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:15.336702 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98077f1a-37d2-4c73-a955-92b88183f999 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:15.349089 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 945/1868] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:51:15 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:51:15.357424 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98077f1a-37d2-4c73-a955-92b88183f999 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:15.375925 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98077f1a-37d2-4c73-a955-92b88183f999 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:15.385535 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871b3022-6280-4717-8fa8-92a1d7465d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:15.386865 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b3022-6280-4717-8fa8-92a1d7465d67 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:15.387350 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b3022-6280-4717-8fa8-92a1d7465d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:15.387741 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b3022-6280-4717-8fa8-92a1d7465d67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:15.389022 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871b3022-6280-4717-8fa8-92a1d7465d67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:15.391759 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871b3022-6280-4717-8fa8-92a1d7465d67 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:15.405593 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87667d6-41bb-48e5-a054-b507e828e998 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:15.413979 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 946/1869] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:51:15 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:51:15.414262 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6418ca-d150-45b4-a5f6-59db3abf5900 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:15.414836 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6418ca-d150-45b4-a5f6-59db3abf5900 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:15.415244 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6418ca-d150-45b4-a5f6-59db3abf5900 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:15.415526 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6418ca-d150-45b4-a5f6-59db3abf5900 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:15.416471 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871b3022-6280-4717-8fa8-92a1d7465d67 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:15.427198 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6418ca-d150-45b4-a5f6-59db3abf5900 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:15.429275 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6418ca-d150-45b4-a5f6-59db3abf5900 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:15.430454 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871b3022-6280-4717-8fa8-92a1d7465d67 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:15.436161 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7d820c-209a-4072-97ed-6b88f3bd1f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:15.437058 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7d820c-209a-4072-97ed-6b88f3bd1f5c admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:15.437334 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7d820c-209a-4072-97ed-6b88f3bd1f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:15.437642 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7d820c-209a-4072-97ed-6b88f3bd1f5c admin admin] PATH_INFO: `/v3/projects/1c88730100ff482687f1d3799498b863/users/6553e14643484a9ea066e3febe6ec989/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:15.440487 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 924/1870] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:51:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:51:15.455056 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7d820c-209a-4072-97ed-6b88f3bd1f5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c88730100ff482687f1d3799498b863, user_id=6553e14643484a9ea066e3febe6ec989, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:15.457875 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7d820c-209a-4072-97ed-6b88f3bd1f5c admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:15.551193 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 947/1871] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:51:15 2022] PUT /identity/v3/projects/1c88730100ff482687f1d3799498b863/users/6553e14643484a9ea066e3febe6ec989/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:15.563510 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc49e8a-cfdb-4639-9282-7e49a62060c0 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:15.563800 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc49e8a-cfdb-4639-9282-7e49a62060c0 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:15.564078 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc49e8a-cfdb-4639-9282-7e49a62060c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:15.637852 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-6bc49e8a-cfdb-4639-9282-7e49a62060c0 None None] MFA Rules not processed for user `6553e14643484a9ea066e3febe6ec989`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:15.641744 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-6bc49e8a-cfdb-4639-9282-7e49a62060c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:15.745458 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:15.772243 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 925/1872] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:51:15 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:51:15.777076 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7d820c-209a-4072-97ed-6b88f3bd1f5c admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:15.858441 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:15.886523 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7d820c-209a-4072-97ed-6b88f3bd1f5c admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:15.894995 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4a01f1-3031-4865-9a95-998af01b42a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:15.901561 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4a01f1-3031-4865-9a95-998af01b42a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:15.901911 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4a01f1-3031-4865-9a95-998af01b42a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:15.902626 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4a01f1-3031-4865-9a95-998af01b42a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:15.902897 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4a01f1-3031-4865-9a95-998af01b42a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:15.904955 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4a01f1-3031-4865-9a95-998af01b42a8 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:16.045506 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 948/1873] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:51:15 2022] POST /identity/v3/projects => generated 365 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:51:16.050939 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc49e8a-cfdb-4639-9282-7e49a62060c0 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:16.063058 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc49e8a-cfdb-4639-9282-7e49a62060c0 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:16.081912 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4a01f1-3031-4865-9a95-998af01b42a8 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:16.083643 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-ce4a01f1-3031-4865-9a95-998af01b42a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:16.189744 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64b2c34-e760-4137-a801-3fc994405879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:16.191771 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b2c34-e760-4137-a801-3fc994405879 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:16.192740 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b2c34-e760-4137-a801-3fc994405879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:16.194396 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b2c34-e760-4137-a801-3fc994405879 admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:16.196929 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64b2c34-e760-4137-a801-3fc994405879 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:16.199069 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4a01f1-3031-4865-9a95-998af01b42a8 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:16.201257 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64b2c34-e760-4137-a801-3fc994405879 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:16.212707 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a569e595-0369-4d39-90ad-e9f7b68aa647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:16.216807 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a569e595-0369-4d39-90ad-e9f7b68aa647 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:16.217102 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a569e595-0369-4d39-90ad-e9f7b68aa647 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:16.217445 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a569e595-0369-4d39-90ad-e9f7b68aa647 admin admin] PATH_INFO: `/v3/projects/11b1b9f71ff64d5ca24d195ca80eab2f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:16.218925 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a569e595-0369-4d39-90ad-e9f7b68aa647 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11b1b9f71ff64d5ca24d195ca80eab2f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:16.221267 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a569e595-0369-4d39-90ad-e9f7b68aa647 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:16.350903 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 926/1874] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:51:16 2022] POST /identity/v3/users => generated 289 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:51:16.354031 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64b2c34-e760-4137-a801-3fc994405879 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:16.370411 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64b2c34-e760-4137-a801-3fc994405879 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:16.375574 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00ac747-8b05-4de2-afc2-fb5b716c1368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:16.376322 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00ac747-8b05-4de2-afc2-fb5b716c1368 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:16.376613 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00ac747-8b05-4de2-afc2-fb5b716c1368 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:16.376895 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00ac747-8b05-4de2-afc2-fb5b716c1368 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:16.377633 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00ac747-8b05-4de2-afc2-fb5b716c1368 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:16.379382 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00ac747-8b05-4de2-afc2-fb5b716c1368 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:16.392930 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 927/1875] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:51:16 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:51:16.397957 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00ac747-8b05-4de2-afc2-fb5b716c1368 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:16.408495 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00ac747-8b05-4de2-afc2-fb5b716c1368 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:16.413904 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dab5418-c030-44c6-b9c2-84df0909bb4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:16.415030 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dab5418-c030-44c6-b9c2-84df0909bb4b admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:16.415317 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dab5418-c030-44c6-b9c2-84df0909bb4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:16.415598 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dab5418-c030-44c6-b9c2-84df0909bb4b admin admin] PATH_INFO: `/v3/projects/eb48138627b94bb3b067b423a31c7aea/users/041ac87aa9754673812ee5c9bd6cc8c0/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:16.434335 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dab5418-c030-44c6-b9c2-84df0909bb4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb48138627b94bb3b067b423a31c7aea, user_id=041ac87aa9754673812ee5c9bd6cc8c0, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:16.437075 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dab5418-c030-44c6-b9c2-84df0909bb4b admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:16.438229 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-a569e595-0369-4d39-90ad-e9f7b68aa647 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11b1b9f71ff64d5ca24d195ca80eab2f', 'request_id': 'req-a569e595-0369-4d39-90ad-e9f7b68aa647'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:16.447457 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-a569e595-0369-4d39-90ad-e9f7b68aa647 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11b1b9f71ff64d5ca24d195ca80eab2f', 'request_id': 'req-a569e595-0369-4d39-90ad-e9f7b68aa647'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:16.525022 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-a569e595-0369-4d39-90ad-e9f7b68aa647 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11b1b9f71ff64d5ca24d195ca80eab2f', 'request_id': 'req-a569e595-0369-4d39-90ad-e9f7b68aa647'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:16.544535 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 928/1876] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:51:16 2022] PUT /identity/v3/projects/eb48138627b94bb3b067b423a31c7aea/users/041ac87aa9754673812ee5c9bd6cc8c0/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:51:16.548315 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dab5418-c030-44c6-b9c2-84df0909bb4b admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:16.562564 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-a569e595-0369-4d39-90ad-e9f7b68aa647 admin admin] The token cache is being invalidate because project 11b1b9f71ff64d5ca24d195ca80eab2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:51:16.562940 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-a569e595-0369-4d39-90ad-e9f7b68aa647 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:16.564514 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-a569e595-0369-4d39-90ad-e9f7b68aa647 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:16.572184 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 949/1877] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:51:16 2022] DELETE /identity/v3/projects/11b1b9f71ff64d5ca24d195ca80eab2f => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:16.573044 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a569e595-0369-4d39-90ad-e9f7b68aa647 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:16.574490 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a569e595-0369-4d39-90ad-e9f7b68aa647 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:16.648878 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:16.676827 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a569e595-0369-4d39-90ad-e9f7b68aa647 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:16.678183 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dab5418-c030-44c6-b9c2-84df0909bb4b admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:16.684130 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050b039b-69ba-4103-89a0-86c8ebe568b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:16.684520 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5c57fc-fd4b-4e7e-89ad-40e6f0abb508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:16.685037 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b039b-69ba-4103-89a0-86c8ebe568b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:16.685221 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c57fc-fd4b-4e7e-89ad-40e6f0abb508 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:16.685412 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b039b-69ba-4103-89a0-86c8ebe568b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:16.685562 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c57fc-fd4b-4e7e-89ad-40e6f0abb508 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:16.685709 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b039b-69ba-4103-89a0-86c8ebe568b4 admin admin] PATH_INFO: `/v3/users/48433e0887a84e3ca10f507c3dbb6b29` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:16.685861 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c57fc-fd4b-4e7e-89ad-40e6f0abb508 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:16.686538 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5c57fc-fd4b-4e7e-89ad-40e6f0abb508 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:16.686884 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050b039b-69ba-4103-89a0-86c8ebe568b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48433e0887a84e3ca10f507c3dbb6b29)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:16.688340 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5c57fc-fd4b-4e7e-89ad-40e6f0abb508 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:16.688871 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050b039b-69ba-4103-89a0-86c8ebe568b4 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:16.691115 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:16.700265 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 929/1878] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:51:16 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:51:16.704354 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5c57fc-fd4b-4e7e-89ad-40e6f0abb508 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:16.706181 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-ea5c57fc-fd4b-4e7e-89ad-40e6f0abb508 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:16.720244 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5c57fc-fd4b-4e7e-89ad-40e6f0abb508 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:16.725165 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60f90b7-351c-4985-80d8-896086fe2aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:16.726109 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60f90b7-351c-4985-80d8-896086fe2aea admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:16.726616 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60f90b7-351c-4985-80d8-896086fe2aea admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:16.727081 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60f90b7-351c-4985-80d8-896086fe2aea admin admin] PATH_INFO: `/v3/projects/eb48138627b94bb3b067b423a31c7aea/users/041ac87aa9754673812ee5c9bd6cc8c0/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:16.735726 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60f90b7-351c-4985-80d8-896086fe2aea admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb48138627b94bb3b067b423a31c7aea, user_id=041ac87aa9754673812ee5c9bd6cc8c0, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:16.739314 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60f90b7-351c-4985-80d8-896086fe2aea admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:16.752700 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 930/1879] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:51:16 2022] PUT /identity/v3/projects/eb48138627b94bb3b067b423a31c7aea/users/041ac87aa9754673812ee5c9bd6cc8c0/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:51:16.764118 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53434ce0-2d34-4aa4-b4bd-aebd26b8c231 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:16.764408 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53434ce0-2d34-4aa4-b4bd-aebd26b8c231 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:16.764830 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53434ce0-2d34-4aa4-b4bd-aebd26b8c231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:16.811224 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-53434ce0-2d34-4aa4-b4bd-aebd26b8c231 None None] MFA Rules not processed for user `041ac87aa9754673812ee5c9bd6cc8c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:16.812569 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-53434ce0-2d34-4aa4-b4bd-aebd26b8c231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:16.890355 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 931/1880] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:51:16 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:51:16.906221 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53434ce0-2d34-4aa4-b4bd-aebd26b8c231 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:16.907645 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-53434ce0-2d34-4aa4-b4bd-aebd26b8c231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:16.972882 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-050b039b-69ba-4103-89a0-86c8ebe568b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48433e0887a84e3ca10f507c3dbb6b29'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:16.974519 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-050b039b-69ba-4103-89a0-86c8ebe568b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48433e0887a84e3ca10f507c3dbb6b29'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:16.976811 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-050b039b-69ba-4103-89a0-86c8ebe568b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48433e0887a84e3ca10f507c3dbb6b29'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:16.979933 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53434ce0-2d34-4aa4-b4bd-aebd26b8c231 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:16.985143 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84da8991-55b0-4d8c-a782-d61772394fff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:16.985810 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84da8991-55b0-4d8c-a782-d61772394fff service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:16.986096 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84da8991-55b0-4d8c-a782-d61772394fff service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:16.986376 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84da8991-55b0-4d8c-a782-d61772394fff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:16.988314 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-84da8991-55b0-4d8c-a782-d61772394fff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:17.003083 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84da8991-55b0-4d8c-a782-d61772394fff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:17.004587 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84da8991-55b0-4d8c-a782-d61772394fff service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:17.016537 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 932/1881] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:51:16 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:51:17.066048 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-050b039b-69ba-4103-89a0-86c8ebe568b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48433e0887a84e3ca10f507c3dbb6b29'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:17.080769 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-050b039b-69ba-4103-89a0-86c8ebe568b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48433e0887a84e3ca10f507c3dbb6b29'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:17.095076 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 950/1882] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:51:16 2022] DELETE /identity/v3/users/48433e0887a84e3ca10f507c3dbb6b29 => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:17.675433 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84da8991-55b0-4d8c-a782-d61772394fff service cinder] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:17.677161 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-84da8991-55b0-4d8c-a782-d61772394fff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:17.747576 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:17.770186 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84da8991-55b0-4d8c-a782-d61772394fff service cinder] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:17.968920 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcaab8c-c819-4d32-b34b-fb5ecab70def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:17.969697 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcaab8c-c819-4d32-b34b-fb5ecab70def admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:17.969971 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcaab8c-c819-4d32-b34b-fb5ecab70def admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:17.970250 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcaab8c-c819-4d32-b34b-fb5ecab70def admin admin] PATH_INFO: `/v3/projects/e9fe1ff2fa2f4b32b74477c6e14cad5d` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:17.971530 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcaab8c-c819-4d32-b34b-fb5ecab70def admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9fe1ff2fa2f4b32b74477c6e14cad5d)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:17.973506 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcaab8c-c819-4d32-b34b-fb5ecab70def admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:18.085177 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-cdcaab8c-c819-4d32-b34b-fb5ecab70def admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9fe1ff2fa2f4b32b74477c6e14cad5d', 'request_id': 'req-cdcaab8c-c819-4d32-b34b-fb5ecab70def'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:18.180678 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-cdcaab8c-c819-4d32-b34b-fb5ecab70def admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9fe1ff2fa2f4b32b74477c6e14cad5d', 'request_id': 'req-cdcaab8c-c819-4d32-b34b-fb5ecab70def'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:18.188587 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-cdcaab8c-c819-4d32-b34b-fb5ecab70def admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9fe1ff2fa2f4b32b74477c6e14cad5d', 'request_id': 'req-cdcaab8c-c819-4d32-b34b-fb5ecab70def'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:18.209534 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-cdcaab8c-c819-4d32-b34b-fb5ecab70def admin admin] The token cache is being invalidate because project e9fe1ff2fa2f4b32b74477c6e14cad5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:51:18.211077 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-cdcaab8c-c819-4d32-b34b-fb5ecab70def admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:18.214029 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-cdcaab8c-c819-4d32-b34b-fb5ecab70def admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:18.217212 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 933/1883] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:51:17 2022] DELETE /identity/v3/projects/e9fe1ff2fa2f4b32b74477c6e14cad5d => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:18.224458 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d7284b-8dd7-4736-8ff7-0903886ffc59 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:18.225197 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d7284b-8dd7-4736-8ff7-0903886ffc59 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:18.225197 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d7284b-8dd7-4736-8ff7-0903886ffc59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:18.247127 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-26d7284b-8dd7-4736-8ff7-0903886ffc59 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:18.248647 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-26d7284b-8dd7-4736-8ff7-0903886ffc59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:18.328535 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 951/1884] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:51:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:18.336934 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4507f165-b58d-44b7-95eb-788e11047a75 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:18.337224 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4507f165-b58d-44b7-95eb-788e11047a75 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:18.337528 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4507f165-b58d-44b7-95eb-788e11047a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:18.358932 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-4507f165-b58d-44b7-95eb-788e11047a75 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:18.360146 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-4507f165-b58d-44b7-95eb-788e11047a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:18.370095 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 934/1885] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:51:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:18.375779 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d7284b-8dd7-4736-8ff7-0903886ffc59 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:18.388863 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d7284b-8dd7-4736-8ff7-0903886ffc59 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:18.395601 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64acdba-3e5b-414b-98b7-b01356c0d357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:18.396365 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64acdba-3e5b-414b-98b7-b01356c0d357 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:18.396965 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64acdba-3e5b-414b-98b7-b01356c0d357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:18.397361 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64acdba-3e5b-414b-98b7-b01356c0d357 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:18.398799 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64acdba-3e5b-414b-98b7-b01356c0d357 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:51:18.399306 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64acdba-3e5b-414b-98b7-b01356c0d357 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:18.402469 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64acdba-3e5b-414b-98b7-b01356c0d357 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:18.417039 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 952/1886] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jun 15 14:51:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:51:18.422088 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4507f165-b58d-44b7-95eb-788e11047a75 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:18.432560 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4507f165-b58d-44b7-95eb-788e11047a75 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:18.437374 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7645d2da-d96b-425e-8c71-7bc2748291ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:18.438039 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7645d2da-d96b-425e-8c71-7bc2748291ae admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:18.438342 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7645d2da-d96b-425e-8c71-7bc2748291ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:18.438608 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7645d2da-d96b-425e-8c71-7bc2748291ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:18.439310 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7645d2da-d96b-425e-8c71-7bc2748291ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:18.441301 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7645d2da-d96b-425e-8c71-7bc2748291ae admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:18.561297 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 935/1887] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:51:18 2022] POST /identity/v3/projects => generated 387 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:51:18.574470 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64acdba-3e5b-414b-98b7-b01356c0d357 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:18.581601 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64acdba-3e5b-414b-98b7-b01356c0d357 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:18.660607 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6204ec31-d4cf-4487-8f10-5a1e8dc19934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:18.661518 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6204ec31-d4cf-4487-8f10-5a1e8dc19934 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:18.662113 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6204ec31-d4cf-4487-8f10-5a1e8dc19934 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:18.662530 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6204ec31-d4cf-4487-8f10-5a1e8dc19934 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:18.663394 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6204ec31-d4cf-4487-8f10-5a1e8dc19934 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:18.667566 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6204ec31-d4cf-4487-8f10-5a1e8dc19934 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:18.680860 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:18.770815 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 953/1888] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:51:18 2022] POST /identity/v3/users => generated 294 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:51:18.776512 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7645d2da-d96b-425e-8c71-7bc2748291ae admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:18.789982 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7645d2da-d96b-425e-8c71-7bc2748291ae admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:18.796188 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737f5b38-a882-4862-9f10-be17fcd465fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:18.796984 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737f5b38-a882-4862-9f10-be17fcd465fe admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:18.797319 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737f5b38-a882-4862-9f10-be17fcd465fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:18.797555 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737f5b38-a882-4862-9f10-be17fcd465fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:18.798287 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737f5b38-a882-4862-9f10-be17fcd465fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:18.800191 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737f5b38-a882-4862-9f10-be17fcd465fe admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:18.825452 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 936/1889] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:51:18 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:51:18.828565 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6204ec31-d4cf-4487-8f10-5a1e8dc19934 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:18.842870 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6204ec31-d4cf-4487-8f10-5a1e8dc19934 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:18.848348 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9350d04-09cf-49aa-b191-11bfb614d2d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:18.849197 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9350d04-09cf-49aa-b191-11bfb614d2d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:18.849430 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9350d04-09cf-49aa-b191-11bfb614d2d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:18.849748 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9350d04-09cf-49aa-b191-11bfb614d2d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:18.850492 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9350d04-09cf-49aa-b191-11bfb614d2d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:18.852798 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9350d04-09cf-49aa-b191-11bfb614d2d5 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:18.865572 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 954/1890] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:51:18 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:51:18.869759 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737f5b38-a882-4862-9f10-be17fcd465fe admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:18.880447 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737f5b38-a882-4862-9f10-be17fcd465fe admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:18.886322 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbec0876-8b6a-414a-87fb-5cd11b29df40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:18.887260 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec0876-8b6a-414a-87fb-5cd11b29df40 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:18.887543 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec0876-8b6a-414a-87fb-5cd11b29df40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:18.887819 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec0876-8b6a-414a-87fb-5cd11b29df40 admin admin] PATH_INFO: `/v3/projects/4d9eb2fe75154c3da36988eef26588b7/users/d9130f13b7874c68a36add1a5e0b7189/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:18.905522 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbec0876-8b6a-414a-87fb-5cd11b29df40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d9eb2fe75154c3da36988eef26588b7, user_id=d9130f13b7874c68a36add1a5e0b7189, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:18.908282 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbec0876-8b6a-414a-87fb-5cd11b29df40 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:18.991223 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 937/1891] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:51:18 2022] PUT /identity/v3/projects/4d9eb2fe75154c3da36988eef26588b7/users/d9130f13b7874c68a36add1a5e0b7189/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:19.009633 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44166dc-5fb7-4222-923f-1831e44d59a7 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:19.009926 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44166dc-5fb7-4222-923f-1831e44d59a7 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:19.010203 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44166dc-5fb7-4222-923f-1831e44d59a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:19.054332 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-a44166dc-5fb7-4222-923f-1831e44d59a7 None None] MFA Rules not processed for user `d9130f13b7874c68a36add1a5e0b7189`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:19.057468 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a44166dc-5fb7-4222-923f-1831e44d59a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:19.140171 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 955/1892] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:51:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:19.144738 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbec0876-8b6a-414a-87fb-5cd11b29df40 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:19.174399 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:19.211672 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbec0876-8b6a-414a-87fb-5cd11b29df40 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:19.216719 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e129742c-5075-4952-9b71-6251da5f4bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:19.217685 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e129742c-5075-4952-9b71-6251da5f4bbb admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:19.217971 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e129742c-5075-4952-9b71-6251da5f4bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:19.218369 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e129742c-5075-4952-9b71-6251da5f4bbb admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:19.218994 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e129742c-5075-4952-9b71-6251da5f4bbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:19.223480 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e129742c-5075-4952-9b71-6251da5f4bbb admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:19.360981 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 938/1893] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:51:19 2022] POST /identity/v3/projects => generated 381 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:51:19.371969 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44166dc-5fb7-4222-923f-1831e44d59a7 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:19.388246 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44166dc-5fb7-4222-923f-1831e44d59a7 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:19.459424 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50638fc0-617a-4d1e-80f6-972d0001238e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:19.460138 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50638fc0-617a-4d1e-80f6-972d0001238e admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:19.460533 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50638fc0-617a-4d1e-80f6-972d0001238e admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:19.460818 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50638fc0-617a-4d1e-80f6-972d0001238e admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:19.461722 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50638fc0-617a-4d1e-80f6-972d0001238e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:19.463658 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50638fc0-617a-4d1e-80f6-972d0001238e admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:19.561719 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 956/1894] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:51:19 2022] POST /identity/v3/users => generated 297 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:51:19.565711 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e129742c-5075-4952-9b71-6251da5f4bbb admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:19.577338 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e129742c-5075-4952-9b71-6251da5f4bbb admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:19.582917 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4105a8d2-5e44-496d-84f4-a6c9b66401ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:19.583585 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4105a8d2-5e44-496d-84f4-a6c9b66401ca admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:19.583962 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4105a8d2-5e44-496d-84f4-a6c9b66401ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:19.584151 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4105a8d2-5e44-496d-84f4-a6c9b66401ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:19.588271 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4105a8d2-5e44-496d-84f4-a6c9b66401ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:19.590372 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4105a8d2-5e44-496d-84f4-a6c9b66401ca admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:19.612478 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 939/1895] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:51:19 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:51:19.613554 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50638fc0-617a-4d1e-80f6-972d0001238e admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:19.628889 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50638fc0-617a-4d1e-80f6-972d0001238e admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:19.639697 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0259a260-b5cd-4042-8758-3d2be02c9587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:19.641635 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0259a260-b5cd-4042-8758-3d2be02c9587 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:19.641925 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0259a260-b5cd-4042-8758-3d2be02c9587 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:19.642203 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0259a260-b5cd-4042-8758-3d2be02c9587 admin admin] PATH_INFO: `/v3/projects/cbdd01a92bf642b39ab0fccccfb6aee2/users/7749b046cf424039bdd80680f6ae57ca/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:19.659437 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0259a260-b5cd-4042-8758-3d2be02c9587 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbdd01a92bf642b39ab0fccccfb6aee2, user_id=7749b046cf424039bdd80680f6ae57ca, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:19.662426 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0259a260-b5cd-4042-8758-3d2be02c9587 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:20.015704 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 957/1896] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:51:19 2022] PUT /identity/v3/projects/cbdd01a92bf642b39ab0fccccfb6aee2/users/7749b046cf424039bdd80680f6ae57ca/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:20.023352 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4105a8d2-5e44-496d-84f4-a6c9b66401ca admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:20.034011 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4105a8d2-5e44-496d-84f4-a6c9b66401ca admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:20.111261 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe97c01e-25e7-48c1-a922-0fd3b008e2b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:20.111991 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe97c01e-25e7-48c1-a922-0fd3b008e2b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:20.112309 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe97c01e-25e7-48c1-a922-0fd3b008e2b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:20.112580 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe97c01e-25e7-48c1-a922-0fd3b008e2b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:20.113355 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe97c01e-25e7-48c1-a922-0fd3b008e2b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:20.114970 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe97c01e-25e7-48c1-a922-0fd3b008e2b0 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:20.127666 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 940/1897] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:51:20 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:51:20.132929 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0259a260-b5cd-4042-8758-3d2be02c9587 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:20.148600 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0259a260-b5cd-4042-8758-3d2be02c9587 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:20.154275 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a28948-74c9-49c3-8ea9-fff7d80efb08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:20.156998 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a28948-74c9-49c3-8ea9-fff7d80efb08 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:20.157876 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a28948-74c9-49c3-8ea9-fff7d80efb08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:20.158602 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a28948-74c9-49c3-8ea9-fff7d80efb08 admin admin] PATH_INFO: `/v3/projects/cbdd01a92bf642b39ab0fccccfb6aee2/users/7749b046cf424039bdd80680f6ae57ca/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:20.162716 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a28948-74c9-49c3-8ea9-fff7d80efb08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbdd01a92bf642b39ab0fccccfb6aee2, user_id=7749b046cf424039bdd80680f6ae57ca, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:20.165742 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a28948-74c9-49c3-8ea9-fff7d80efb08 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:20.177558 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 958/1898] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:51:20 2022] PUT /identity/v3/projects/cbdd01a92bf642b39ab0fccccfb6aee2/users/7749b046cf424039bdd80680f6ae57ca/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:20.188837 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9490b69-3b9e-4648-b0a6-1ad808d53cc4 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:20.197410 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9490b69-3b9e-4648-b0a6-1ad808d53cc4 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:20.197410 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9490b69-3b9e-4648-b0a6-1ad808d53cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:20.219008 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-a9490b69-3b9e-4648-b0a6-1ad808d53cc4 None None] MFA Rules not processed for user `7749b046cf424039bdd80680f6ae57ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:20.225405 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-a9490b69-3b9e-4648-b0a6-1ad808d53cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:20.320121 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 941/1899] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:51:20 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:20.328238 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a28948-74c9-49c3-8ea9-fff7d80efb08 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:20.329998 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a7a28948-74c9-49c3-8ea9-fff7d80efb08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:20.365474 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:20.395147 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a28948-74c9-49c3-8ea9-fff7d80efb08 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:20.399815 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5782849-6640-4679-bf05-104a46bdb0cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:20.400505 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5782849-6640-4679-bf05-104a46bdb0cc service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:20.400790 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5782849-6640-4679-bf05-104a46bdb0cc service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:20.401065 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5782849-6640-4679-bf05-104a46bdb0cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:20.411116 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5782849-6640-4679-bf05-104a46bdb0cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:20.412590 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5782849-6640-4679-bf05-104a46bdb0cc service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:20.425083 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 959/1900] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:51:20 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:51:20.661802 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9490b69-3b9e-4648-b0a6-1ad808d53cc4 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:20.663811 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-a9490b69-3b9e-4648-b0a6-1ad808d53cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:20.691224 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:20.751267 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9490b69-3b9e-4648-b0a6-1ad808d53cc4 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:20.756396 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502c7bb6-e298-49f1-a25c-1c28904dd624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:20.757393 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502c7bb6-e298-49f1-a25c-1c28904dd624 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:20.757823 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502c7bb6-e298-49f1-a25c-1c28904dd624 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:20.758211 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502c7bb6-e298-49f1-a25c-1c28904dd624 admin admin] PATH_INFO: `/v3/users/6553e14643484a9ea066e3febe6ec989` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:20.759497 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502c7bb6-e298-49f1-a25c-1c28904dd624 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6553e14643484a9ea066e3febe6ec989)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:20.761717 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502c7bb6-e298-49f1-a25c-1c28904dd624 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:20.977635 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-502c7bb6-e298-49f1-a25c-1c28904dd624 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6553e14643484a9ea066e3febe6ec989'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:20.978526 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-502c7bb6-e298-49f1-a25c-1c28904dd624 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6553e14643484a9ea066e3febe6ec989'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:20.987829 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-502c7bb6-e298-49f1-a25c-1c28904dd624 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6553e14643484a9ea066e3febe6ec989'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:20.989345 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-502c7bb6-e298-49f1-a25c-1c28904dd624 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6553e14643484a9ea066e3febe6ec989'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:21.060701 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-502c7bb6-e298-49f1-a25c-1c28904dd624 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6553e14643484a9ea066e3febe6ec989'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:21.098516 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 942/1901] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:51:20 2022] DELETE /identity/v3/users/6553e14643484a9ea066e3febe6ec989 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:21.103745 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be9c7f5-f9f3-420d-9470-e94fdfd27081 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:21.104133 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be9c7f5-f9f3-420d-9470-e94fdfd27081 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:21.104533 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be9c7f5-f9f3-420d-9470-e94fdfd27081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:21.122406 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-1be9c7f5-f9f3-420d-9470-e94fdfd27081 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:21.123764 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-1be9c7f5-f9f3-420d-9470-e94fdfd27081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:21.203671 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 960/1902] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:51:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:51:21.208128 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502c7bb6-e298-49f1-a25c-1c28904dd624 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:21.209574 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-502c7bb6-e298-49f1-a25c-1c28904dd624 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:21.286733 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502c7bb6-e298-49f1-a25c-1c28904dd624 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:21.290341 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615a787f-9b54-4847-ad13-4771cf362a99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:21.290994 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a787f-9b54-4847-ad13-4771cf362a99 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:21.291272 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a787f-9b54-4847-ad13-4771cf362a99 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:21.291541 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a787f-9b54-4847-ad13-4771cf362a99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:21.301607 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615a787f-9b54-4847-ad13-4771cf362a99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:21.303073 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615a787f-9b54-4847-ad13-4771cf362a99 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:21.313681 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 943/1903] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:51:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:51:21.789361 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be9c7f5-f9f3-420d-9470-e94fdfd27081 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:21.791580 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-1be9c7f5-f9f3-420d-9470-e94fdfd27081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:21.811484 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be9c7f5-f9f3-420d-9470-e94fdfd27081 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:21.824472 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4e4b3b-5e78-4c36-850b-01cc85df6df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:21.825819 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4e4b3b-5e78-4c36-850b-01cc85df6df8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:21.826107 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4e4b3b-5e78-4c36-850b-01cc85df6df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:21.826381 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4e4b3b-5e78-4c36-850b-01cc85df6df8 admin admin] PATH_INFO: `/v3/projects/1c88730100ff482687f1d3799498b863` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:21.827619 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4e4b3b-5e78-4c36-850b-01cc85df6df8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c88730100ff482687f1d3799498b863)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:21.829635 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4e4b3b-5e78-4c36-850b-01cc85df6df8 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:21.885525 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615a787f-9b54-4847-ad13-4771cf362a99 service neutron] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:21.887431 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-615a787f-9b54-4847-ad13-4771cf362a99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:21.901481 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615a787f-9b54-4847-ad13-4771cf362a99 service neutron] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:21.909715 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bbce58-ac7a-44c0-8a10-9ba7657e45bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:21.911447 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bbce58-ac7a-44c0-8a10-9ba7657e45bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:21.912106 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bbce58-ac7a-44c0-8a10-9ba7657e45bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:21.912608 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bbce58-ac7a-44c0-8a10-9ba7657e45bb admin admin] PATH_INFO: `/v3/users/d9130f13b7874c68a36add1a5e0b7189` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:21.915174 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bbce58-ac7a-44c0-8a10-9ba7657e45bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9130f13b7874c68a36add1a5e0b7189)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:21.919254 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bbce58-ac7a-44c0-8a10-9ba7657e45bb admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:21.953748 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-4d4e4b3b-5e78-4c36-850b-01cc85df6df8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c88730100ff482687f1d3799498b863', 'request_id': 'req-4d4e4b3b-5e78-4c36-850b-01cc85df6df8'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:21.960124 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-4d4e4b3b-5e78-4c36-850b-01cc85df6df8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c88730100ff482687f1d3799498b863', 'request_id': 'req-4d4e4b3b-5e78-4c36-850b-01cc85df6df8'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:22.053906 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-4d4e4b3b-5e78-4c36-850b-01cc85df6df8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c88730100ff482687f1d3799498b863', 'request_id': 'req-4d4e4b3b-5e78-4c36-850b-01cc85df6df8'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:22.069015 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-4d4e4b3b-5e78-4c36-850b-01cc85df6df8 admin admin] The token cache is being invalidate because project 1c88730100ff482687f1d3799498b863 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:51:22.069301 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-4d4e4b3b-5e78-4c36-850b-01cc85df6df8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:22.070677 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-4d4e4b3b-5e78-4c36-850b-01cc85df6df8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:22.072309 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 961/1904] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:51:21 2022] DELETE /identity/v3/projects/1c88730100ff482687f1d3799498b863 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:22.076347 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4e4b3b-5e78-4c36-850b-01cc85df6df8 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:22.077721 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-4d4e4b3b-5e78-4c36-850b-01cc85df6df8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:22.094598 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:22.105133 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-f5bbce58-ac7a-44c0-8a10-9ba7657e45bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9130f13b7874c68a36add1a5e0b7189'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:22.105887 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-f5bbce58-ac7a-44c0-8a10-9ba7657e45bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9130f13b7874c68a36add1a5e0b7189'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:22.117390 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-f5bbce58-ac7a-44c0-8a10-9ba7657e45bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9130f13b7874c68a36add1a5e0b7189'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:22.118568 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-f5bbce58-ac7a-44c0-8a10-9ba7657e45bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9130f13b7874c68a36add1a5e0b7189'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:22.149643 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:22.175410 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4e4b3b-5e78-4c36-850b-01cc85df6df8 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:22.175410 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b2d82d-a29e-4b9c-b5a7-fb64d7d31679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:22.175410 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b2d82d-a29e-4b9c-b5a7-fb64d7d31679 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:22.175410 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b2d82d-a29e-4b9c-b5a7-fb64d7d31679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:22.175912 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b2d82d-a29e-4b9c-b5a7-fb64d7d31679 admin admin] PATH_INFO: `/v3/users/041ac87aa9754673812ee5c9bd6cc8c0` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:22.175912 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b2d82d-a29e-4b9c-b5a7-fb64d7d31679 admin admin] RBAC: Authorizing `identity:delete_user(user_id=041ac87aa9754673812ee5c9bd6cc8c0)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:22.177592 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b2d82d-a29e-4b9c-b5a7-fb64d7d31679 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:22.214328 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-f5bbce58-ac7a-44c0-8a10-9ba7657e45bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9130f13b7874c68a36add1a5e0b7189'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:22.235628 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 944/1905] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:51:21 2022] DELETE /identity/v3/users/d9130f13b7874c68a36add1a5e0b7189 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:51:22.240038 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5394a9-73c3-401f-a6d7-e879fda7e755 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:22.244301 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5394a9-73c3-401f-a6d7-e879fda7e755 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:22.244301 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5394a9-73c3-401f-a6d7-e879fda7e755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:22.260512 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-ca5394a9-73c3-401f-a6d7-e879fda7e755 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:22.262107 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-ca5394a9-73c3-401f-a6d7-e879fda7e755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:22.332968 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 945/1906] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:51:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:51:22.343302 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5394a9-73c3-401f-a6d7-e879fda7e755 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:22.346406 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-ca5394a9-73c3-401f-a6d7-e879fda7e755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:22.402081 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-99b2d82d-a29e-4b9c-b5a7-fb64d7d31679 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '041ac87aa9754673812ee5c9bd6cc8c0'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:22.402738 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-99b2d82d-a29e-4b9c-b5a7-fb64d7d31679 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '041ac87aa9754673812ee5c9bd6cc8c0'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:22.403341 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-99b2d82d-a29e-4b9c-b5a7-fb64d7d31679 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '041ac87aa9754673812ee5c9bd6cc8c0'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:22.408698 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5394a9-73c3-401f-a6d7-e879fda7e755 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:22.413029 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9ae1a6-1db3-4719-9c13-aca8d31a6c5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:22.413877 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9ae1a6-1db3-4719-9c13-aca8d31a6c5f service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:22.414162 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9ae1a6-1db3-4719-9c13-aca8d31a6c5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:22.414444 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9ae1a6-1db3-4719-9c13-aca8d31a6c5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:22.416299 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-ee9ae1a6-1db3-4719-9c13-aca8d31a6c5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:22.428277 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9ae1a6-1db3-4719-9c13-aca8d31a6c5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:22.430209 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9ae1a6-1db3-4719-9c13-aca8d31a6c5f service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:22.441123 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 946/1907] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:51:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:51:22.488963 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-99b2d82d-a29e-4b9c-b5a7-fb64d7d31679 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '041ac87aa9754673812ee5c9bd6cc8c0'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:22.504079 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-99b2d82d-a29e-4b9c-b5a7-fb64d7d31679 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '041ac87aa9754673812ee5c9bd6cc8c0'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:22.519770 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 962/1908] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:51:22 2022] DELETE /identity/v3/users/041ac87aa9754673812ee5c9bd6cc8c0 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:51:23.070097 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9ae1a6-1db3-4719-9c13-aca8d31a6c5f service neutron] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:23.071370 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b2d82d-a29e-4b9c-b5a7-fb64d7d31679 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:23.073164 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-99b2d82d-a29e-4b9c-b5a7-fb64d7d31679 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:23.073368 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-ee9ae1a6-1db3-4719-9c13-aca8d31a6c5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:23.089585 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9ae1a6-1db3-4719-9c13-aca8d31a6c5f service neutron] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:23.163454 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:23.171401 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b2d82d-a29e-4b9c-b5a7-fb64d7d31679 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:23.183618 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbb7623-02f6-4408-9dad-72eda6bd0f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:23.184363 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbb7623-02f6-4408-9dad-72eda6bd0f72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:23.184717 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbb7623-02f6-4408-9dad-72eda6bd0f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:23.185118 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbb7623-02f6-4408-9dad-72eda6bd0f72 admin admin] PATH_INFO: `/v3/projects/4d9eb2fe75154c3da36988eef26588b7` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:23.186400 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbb7623-02f6-4408-9dad-72eda6bd0f72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d9eb2fe75154c3da36988eef26588b7)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:23.187098 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fc330c-1e90-42c5-93ac-63d989ffc98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:23.188545 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fc330c-1e90-42c5-93ac-63d989ffc98b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:23.188701 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbb7623-02f6-4408-9dad-72eda6bd0f72 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:23.189058 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fc330c-1e90-42c5-93ac-63d989ffc98b admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:23.189444 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fc330c-1e90-42c5-93ac-63d989ffc98b admin admin] PATH_INFO: `/v3/projects/eb48138627b94bb3b067b423a31c7aea` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:23.191313 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fc330c-1e90-42c5-93ac-63d989ffc98b admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb48138627b94bb3b067b423a31c7aea)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:23.194059 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fc330c-1e90-42c5-93ac-63d989ffc98b admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:23.314521 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-dfbb7623-02f6-4408-9dad-72eda6bd0f72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d9eb2fe75154c3da36988eef26588b7', 'request_id': 'req-dfbb7623-02f6-4408-9dad-72eda6bd0f72'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:23.322656 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-dfbb7623-02f6-4408-9dad-72eda6bd0f72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d9eb2fe75154c3da36988eef26588b7', 'request_id': 'req-dfbb7623-02f6-4408-9dad-72eda6bd0f72'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:23.338574 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-11fc330c-1e90-42c5-93ac-63d989ffc98b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb48138627b94bb3b067b423a31c7aea', 'request_id': 'req-11fc330c-1e90-42c5-93ac-63d989ffc98b'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:23.434872 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-dfbb7623-02f6-4408-9dad-72eda6bd0f72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d9eb2fe75154c3da36988eef26588b7', 'request_id': 'req-dfbb7623-02f6-4408-9dad-72eda6bd0f72'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:23.449621 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-dfbb7623-02f6-4408-9dad-72eda6bd0f72 admin admin] The token cache is being invalidate because project 4d9eb2fe75154c3da36988eef26588b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:51:23.449917 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-dfbb7623-02f6-4408-9dad-72eda6bd0f72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:23.450602 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-dfbb7623-02f6-4408-9dad-72eda6bd0f72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:23.452304 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 963/1909] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:51:23 2022] DELETE /identity/v3/projects/4d9eb2fe75154c3da36988eef26588b7 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:51:23.456673 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbb7623-02f6-4408-9dad-72eda6bd0f72 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:23.458499 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-dfbb7623-02f6-4408-9dad-72eda6bd0f72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:23.467224 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:23.488961 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-11fc330c-1e90-42c5-93ac-63d989ffc98b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb48138627b94bb3b067b423a31c7aea', 'request_id': 'req-11fc330c-1e90-42c5-93ac-63d989ffc98b'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:23.499335 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-11fc330c-1e90-42c5-93ac-63d989ffc98b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb48138627b94bb3b067b423a31c7aea', 'request_id': 'req-11fc330c-1e90-42c5-93ac-63d989ffc98b'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:23.514872 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-11fc330c-1e90-42c5-93ac-63d989ffc98b admin admin] The token cache is being invalidate because project eb48138627b94bb3b067b423a31c7aea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:51:23.515463 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-11fc330c-1e90-42c5-93ac-63d989ffc98b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:23.516257 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-11fc330c-1e90-42c5-93ac-63d989ffc98b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:23.517982 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 947/1910] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:51:23 2022] DELETE /identity/v3/projects/eb48138627b94bb3b067b423a31c7aea => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:23.525378 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a635928d-b9c3-4b24-a800-4c6971f4b8b2 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:23.525764 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a635928d-b9c3-4b24-a800-4c6971f4b8b2 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:23.526150 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a635928d-b9c3-4b24-a800-4c6971f4b8b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:23.546742 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-a635928d-b9c3-4b24-a800-4c6971f4b8b2 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:23.548038 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-a635928d-b9c3-4b24-a800-4c6971f4b8b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:23.562777 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbb7623-02f6-4408-9dad-72eda6bd0f72 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:23.627189 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 948/1911] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:51:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:23.637576 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2529f1f-900f-4ced-ba16-c815abfa679d None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:23.640701 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2529f1f-900f-4ced-ba16-c815abfa679d None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:23.640701 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2529f1f-900f-4ced-ba16-c815abfa679d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:23.640885 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a034bc-ebd8-4b94-ab1f-a71a48498504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:23.641800 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a034bc-ebd8-4b94-ab1f-a71a48498504 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:23.642073 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a034bc-ebd8-4b94-ab1f-a71a48498504 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:23.642339 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a034bc-ebd8-4b94-ab1f-a71a48498504 admin admin] PATH_INFO: `/v3/users/7749b046cf424039bdd80680f6ae57ca` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:23.643563 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a034bc-ebd8-4b94-ab1f-a71a48498504 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7749b046cf424039bdd80680f6ae57ca)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:23.645580 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a034bc-ebd8-4b94-ab1f-a71a48498504 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:23.673169 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-a2529f1f-900f-4ced-ba16-c815abfa679d None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:23.675280 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-a2529f1f-900f-4ced-ba16-c815abfa679d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:23.683401 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 949/1912] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:51:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:51:23.688312 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2529f1f-900f-4ced-ba16-c815abfa679d None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:23.699218 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2529f1f-900f-4ced-ba16-c815abfa679d None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:23.705630 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98712e95-4c49-4110-abf8-9d0f6fa865b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:23.706478 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98712e95-4c49-4110-abf8-9d0f6fa865b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:23.712820 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98712e95-4c49-4110-abf8-9d0f6fa865b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:23.712820 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98712e95-4c49-4110-abf8-9d0f6fa865b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:23.712820 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98712e95-4c49-4110-abf8-9d0f6fa865b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:51:23.712820 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98712e95-4c49-4110-abf8-9d0f6fa865b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:23.712820 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98712e95-4c49-4110-abf8-9d0f6fa865b9 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:23.723347 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 950/1913] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jun 15 14:51:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:51:23.728510 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98712e95-4c49-4110-abf8-9d0f6fa865b9 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:23.738778 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98712e95-4c49-4110-abf8-9d0f6fa865b9 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:23.744273 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af297190-880d-4324-acee-c9d0ac5252e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:23.744940 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af297190-880d-4324-acee-c9d0ac5252e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:23.747058 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af297190-880d-4324-acee-c9d0ac5252e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:23.747058 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af297190-880d-4324-acee-c9d0ac5252e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:23.747058 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af297190-880d-4324-acee-c9d0ac5252e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:23.748178 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af297190-880d-4324-acee-c9d0ac5252e3 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:23.909209 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 951/1914] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:51:23 2022] POST /identity/v3/projects => generated 389 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:51:23.914174 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af297190-880d-4324-acee-c9d0ac5252e3 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:23.966978 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-86a034bc-ebd8-4b94-ab1f-a71a48498504 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7749b046cf424039bdd80680f6ae57ca'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:23.968251 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-86a034bc-ebd8-4b94-ab1f-a71a48498504 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7749b046cf424039bdd80680f6ae57ca'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:23.969156 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-86a034bc-ebd8-4b94-ab1f-a71a48498504 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7749b046cf424039bdd80680f6ae57ca'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:23.974499 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:23.993806 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af297190-880d-4324-acee-c9d0ac5252e3 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:23.998922 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6dd1db-8b84-423c-a7c8-cd6285676758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:23.999601 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6dd1db-8b84-423c-a7c8-cd6285676758 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:23.999879 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6dd1db-8b84-423c-a7c8-cd6285676758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:24.000184 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6dd1db-8b84-423c-a7c8-cd6285676758 admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:24.000920 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6dd1db-8b84-423c-a7c8-cd6285676758 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:24.002807 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6dd1db-8b84-423c-a7c8-cd6285676758 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:24.037270 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-86a034bc-ebd8-4b94-ab1f-a71a48498504 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7749b046cf424039bdd80680f6ae57ca'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:24.054819 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-86a034bc-ebd8-4b94-ab1f-a71a48498504 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7749b046cf424039bdd80680f6ae57ca'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:24.070891 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 964/1915] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:51:23 2022] DELETE /identity/v3/users/7749b046cf424039bdd80680f6ae57ca => generated 0 bytes in 613 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:51:24.130646 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 952/1916] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:51:23 2022] POST /identity/v3/users => generated 295 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:51:24.136728 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a034bc-ebd8-4b94-ab1f-a71a48498504 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:24.138289 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-86a034bc-ebd8-4b94-ab1f-a71a48498504 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:24.219752 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:24.271418 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a034bc-ebd8-4b94-ab1f-a71a48498504 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:24.277065 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c478d62-e1e6-454e-9669-b71a7a8d5132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:24.277768 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c478d62-e1e6-454e-9669-b71a7a8d5132 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:24.278047 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c478d62-e1e6-454e-9669-b71a7a8d5132 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:24.278327 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c478d62-e1e6-454e-9669-b71a7a8d5132 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:24.291232 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c478d62-e1e6-454e-9669-b71a7a8d5132 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:24.293334 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c478d62-e1e6-454e-9669-b71a7a8d5132 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:24.315387 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 965/1917] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:51:24 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:51:24.324839 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6dd1db-8b84-423c-a7c8-cd6285676758 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:24.340129 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6dd1db-8b84-423c-a7c8-cd6285676758 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:24.345598 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a28e774-9790-47e1-8e8f-6b109b2492bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:24.346335 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a28e774-9790-47e1-8e8f-6b109b2492bd admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:24.346619 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a28e774-9790-47e1-8e8f-6b109b2492bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:24.346894 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a28e774-9790-47e1-8e8f-6b109b2492bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:24.347632 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a28e774-9790-47e1-8e8f-6b109b2492bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:24.352830 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a28e774-9790-47e1-8e8f-6b109b2492bd admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:24.372603 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 953/1918] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:51:24 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:51:24.377931 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c478d62-e1e6-454e-9669-b71a7a8d5132 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:24.389797 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c478d62-e1e6-454e-9669-b71a7a8d5132 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:24.394603 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473b6134-a15e-470c-a3ea-8fddc09d5ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:24.395509 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b6134-a15e-470c-a3ea-8fddc09d5ddf admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:24.396009 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b6134-a15e-470c-a3ea-8fddc09d5ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:24.396504 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b6134-a15e-470c-a3ea-8fddc09d5ddf admin admin] PATH_INFO: `/v3/projects/06feb98470404db682c8331ae91c844e/users/82f94ca84f1c42b180ad65dab0c100e8/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:24.412688 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473b6134-a15e-470c-a3ea-8fddc09d5ddf admin admin] RBAC: Authorizing `identity:create_grant(project_id=06feb98470404db682c8331ae91c844e, user_id=82f94ca84f1c42b180ad65dab0c100e8, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:24.416397 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473b6134-a15e-470c-a3ea-8fddc09d5ddf admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:24.500869 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 966/1919] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:51:24 2022] PUT /identity/v3/projects/06feb98470404db682c8331ae91c844e/users/82f94ca84f1c42b180ad65dab0c100e8/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:24.514861 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addc614a-8282-46b8-9a25-8eaed446a6c4 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:24.515725 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addc614a-8282-46b8-9a25-8eaed446a6c4 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:24.516215 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addc614a-8282-46b8-9a25-8eaed446a6c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:24.557641 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-addc614a-8282-46b8-9a25-8eaed446a6c4 None None] MFA Rules not processed for user `82f94ca84f1c42b180ad65dab0c100e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:24.559044 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-addc614a-8282-46b8-9a25-8eaed446a6c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:24.648890 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 954/1920] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:51:24 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:51:24.654543 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473b6134-a15e-470c-a3ea-8fddc09d5ddf admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:24.760666 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473b6134-a15e-470c-a3ea-8fddc09d5ddf admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:24.766539 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda2806b-80c8-4ad0-b70e-d8e8ba650928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:24.767257 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda2806b-80c8-4ad0-b70e-d8e8ba650928 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:24.767543 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda2806b-80c8-4ad0-b70e-d8e8ba650928 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:24.767994 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda2806b-80c8-4ad0-b70e-d8e8ba650928 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:24.769139 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda2806b-80c8-4ad0-b70e-d8e8ba650928 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:24.771484 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda2806b-80c8-4ad0-b70e-d8e8ba650928 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:24.773258 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addc614a-8282-46b8-9a25-8eaed446a6c4 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:24.775325 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-addc614a-8282-46b8-9a25-8eaed446a6c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:24.813945 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addc614a-8282-46b8-9a25-8eaed446a6c4 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:24.824425 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3501aa98-e813-4748-8d4f-55e2ddd6d852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:24.825647 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3501aa98-e813-4748-8d4f-55e2ddd6d852 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:24.826293 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3501aa98-e813-4748-8d4f-55e2ddd6d852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:24.826831 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3501aa98-e813-4748-8d4f-55e2ddd6d852 admin admin] PATH_INFO: `/v3/projects/cbdd01a92bf642b39ab0fccccfb6aee2` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:24.828823 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3501aa98-e813-4748-8d4f-55e2ddd6d852 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbdd01a92bf642b39ab0fccccfb6aee2)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:24.833612 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3501aa98-e813-4748-8d4f-55e2ddd6d852 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:24.875208 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 967/1921] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:51:24 2022] POST /identity/v3/projects => generated 387 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:51:24.880781 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda2806b-80c8-4ad0-b70e-d8e8ba650928 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:24.982616 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda2806b-80c8-4ad0-b70e-d8e8ba650928 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:24.988268 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac95048-6aaa-41f4-9d59-42bfbb3fc78b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:24.989091 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac95048-6aaa-41f4-9d59-42bfbb3fc78b admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:24.989464 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac95048-6aaa-41f4-9d59-42bfbb3fc78b admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:24.989846 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac95048-6aaa-41f4-9d59-42bfbb3fc78b admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:24.990905 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac95048-6aaa-41f4-9d59-42bfbb3fc78b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:24.994296 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac95048-6aaa-41f4-9d59-42bfbb3fc78b admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:25.017336 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-3501aa98-e813-4748-8d4f-55e2ddd6d852 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbdd01a92bf642b39ab0fccccfb6aee2', 'request_id': 'req-3501aa98-e813-4748-8d4f-55e2ddd6d852'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:25.082903 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-3501aa98-e813-4748-8d4f-55e2ddd6d852 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbdd01a92bf642b39ab0fccccfb6aee2', 'request_id': 'req-3501aa98-e813-4748-8d4f-55e2ddd6d852'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:25.090401 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-3501aa98-e813-4748-8d4f-55e2ddd6d852 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbdd01a92bf642b39ab0fccccfb6aee2', 'request_id': 'req-3501aa98-e813-4748-8d4f-55e2ddd6d852'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:25.094779 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 968/1922] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:51:24 2022] POST /identity/v3/users => generated 300 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:51:25.095763 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac95048-6aaa-41f4-9d59-42bfbb3fc78b admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:25.103717 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-3501aa98-e813-4748-8d4f-55e2ddd6d852 admin admin] The token cache is being invalidate because project cbdd01a92bf642b39ab0fccccfb6aee2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:51:25.104019 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-3501aa98-e813-4748-8d4f-55e2ddd6d852 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:25.104730 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-3501aa98-e813-4748-8d4f-55e2ddd6d852 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:25.106336 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 955/1923] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:51:24 2022] DELETE /identity/v3/projects/cbdd01a92bf642b39ab0fccccfb6aee2 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:25.108494 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac95048-6aaa-41f4-9d59-42bfbb3fc78b admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:25.112657 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858441db-6537-40ef-acae-ba074c5bec5e None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:25.113088 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858441db-6537-40ef-acae-ba074c5bec5e None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:25.115940 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858441db-6537-40ef-acae-ba074c5bec5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:25.124909 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:25.140852 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-858441db-6537-40ef-acae-ba074c5bec5e None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:25.142831 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-858441db-6537-40ef-acae-ba074c5bec5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:25.188311 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3276f9e-c18f-4b0f-9c2c-bdce53980524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:25.189159 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3276f9e-c18f-4b0f-9c2c-bdce53980524 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:25.189448 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3276f9e-c18f-4b0f-9c2c-bdce53980524 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:25.189727 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3276f9e-c18f-4b0f-9c2c-bdce53980524 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:25.190478 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3276f9e-c18f-4b0f-9c2c-bdce53980524 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:25.192273 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3276f9e-c18f-4b0f-9c2c-bdce53980524 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:25.199590 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:25.207972 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 969/1924] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:51:25 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:51:25.209060 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3276f9e-c18f-4b0f-9c2c-bdce53980524 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:25.210958 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-f3276f9e-c18f-4b0f-9c2c-bdce53980524 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:25.223573 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3276f9e-c18f-4b0f-9c2c-bdce53980524 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:25.230584 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe60540-58c1-4e21-8cb7-6889fec37802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:25.230750 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 956/1925] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:51:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:51:25.230857 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe60540-58c1-4e21-8cb7-6889fec37802 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:25.230857 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe60540-58c1-4e21-8cb7-6889fec37802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:25.230857 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe60540-58c1-4e21-8cb7-6889fec37802 admin admin] PATH_INFO: `/v3/projects/732fafa3f71f42ea88fbdcd991bcc661/users/75f6a65d4f48496bb6c65704beb8bf86/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:25.238199 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24678e11-67c2-4204-a042-f3740f315518 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:25.238632 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24678e11-67c2-4204-a042-f3740f315518 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:25.239056 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24678e11-67c2-4204-a042-f3740f315518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:25.254612 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe60540-58c1-4e21-8cb7-6889fec37802 admin admin] RBAC: Authorizing `identity:create_grant(project_id=732fafa3f71f42ea88fbdcd991bcc661, user_id=75f6a65d4f48496bb6c65704beb8bf86, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:25.257645 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe60540-58c1-4e21-8cb7-6889fec37802 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:25.262489 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-24678e11-67c2-4204-a042-f3740f315518 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:25.263678 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-24678e11-67c2-4204-a042-f3740f315518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:25.271603 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 957/1926] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:51:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:51:25.279240 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24678e11-67c2-4204-a042-f3740f315518 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:25.290160 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24678e11-67c2-4204-a042-f3740f315518 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:25.294987 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383ec8bf-eed9-4174-8c8b-c720159c3cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:25.295658 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ec8bf-eed9-4174-8c8b-c720159c3cbc admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:25.295942 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ec8bf-eed9-4174-8c8b-c720159c3cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:25.296262 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ec8bf-eed9-4174-8c8b-c720159c3cbc admin admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:25.296987 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383ec8bf-eed9-4174-8c8b-c720159c3cbc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:51:25.297296 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383ec8bf-eed9-4174-8c8b-c720159c3cbc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:25.298933 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383ec8bf-eed9-4174-8c8b-c720159c3cbc admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:25.310967 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 958/1927] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jun 15 14:51:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:51:25.315185 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383ec8bf-eed9-4174-8c8b-c720159c3cbc admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:25.325165 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383ec8bf-eed9-4174-8c8b-c720159c3cbc admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:25.330551 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86e2175-4372-49bb-bbe6-c4e352d3c970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:25.331211 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e2175-4372-49bb-bbe6-c4e352d3c970 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:25.331486 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e2175-4372-49bb-bbe6-c4e352d3c970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:25.331766 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e2175-4372-49bb-bbe6-c4e352d3c970 admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:25.332509 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86e2175-4372-49bb-bbe6-c4e352d3c970 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:25.334428 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86e2175-4372-49bb-bbe6-c4e352d3c970 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:25.345350 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 970/1928] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:51:25 2022] PUT /identity/v3/projects/732fafa3f71f42ea88fbdcd991bcc661/users/75f6a65d4f48496bb6c65704beb8bf86/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:25.349963 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe60540-58c1-4e21-8cb7-6889fec37802 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:25.422622 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe60540-58c1-4e21-8cb7-6889fec37802 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:25.427819 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5527abab-b484-4e89-948d-cc0705c2728c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:25.428539 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5527abab-b484-4e89-948d-cc0705c2728c admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:25.429095 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5527abab-b484-4e89-948d-cc0705c2728c admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:25.429456 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5527abab-b484-4e89-948d-cc0705c2728c admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:25.430196 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5527abab-b484-4e89-948d-cc0705c2728c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:25.431808 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5527abab-b484-4e89-948d-cc0705c2728c admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:25.441171 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 959/1929] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:51:25 2022] POST /identity/v3/projects => generated 385 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:51:25.445481 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86e2175-4372-49bb-bbe6-c4e352d3c970 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:25.454803 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 971/1930] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:51:25 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:51:25.455955 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5527abab-b484-4e89-948d-cc0705c2728c admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:25.470126 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5527abab-b484-4e89-948d-cc0705c2728c admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:25.527203 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86e2175-4372-49bb-bbe6-c4e352d3c970 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:25.532198 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d34eb5-cc8a-4289-b69e-c19150b51135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:25.532890 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d34eb5-cc8a-4289-b69e-c19150b51135 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:25.533180 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d34eb5-cc8a-4289-b69e-c19150b51135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:25.533460 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d34eb5-cc8a-4289-b69e-c19150b51135 admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:25.534207 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d34eb5-cc8a-4289-b69e-c19150b51135 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:25.536158 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d34eb5-cc8a-4289-b69e-c19150b51135 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:25.559756 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf644da-f10c-42ad-98d2-358904ba70b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:25.560842 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf644da-f10c-42ad-98d2-358904ba70b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:25.561149 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf644da-f10c-42ad-98d2-358904ba70b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:25.561529 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf644da-f10c-42ad-98d2-358904ba70b9 admin admin] PATH_INFO: `/v3/projects/732fafa3f71f42ea88fbdcd991bcc661/users/75f6a65d4f48496bb6c65704beb8bf86/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:25.563652 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf644da-f10c-42ad-98d2-358904ba70b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=732fafa3f71f42ea88fbdcd991bcc661, user_id=75f6a65d4f48496bb6c65704beb8bf86, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:25.566423 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf644da-f10c-42ad-98d2-358904ba70b9 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:25.581807 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 972/1931] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:51:25 2022] PUT /identity/v3/projects/732fafa3f71f42ea88fbdcd991bcc661/users/75f6a65d4f48496bb6c65704beb8bf86/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:25.598091 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf22172-9d7a-42c6-a50e-2893f25c23a1 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:25.598376 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf22172-9d7a-42c6-a50e-2893f25c23a1 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:25.598656 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf22172-9d7a-42c6-a50e-2893f25c23a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:25.628063 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-3bf22172-9d7a-42c6-a50e-2893f25c23a1 None None] MFA Rules not processed for user `75f6a65d4f48496bb6c65704beb8bf86`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:25.629362 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-3bf22172-9d7a-42c6-a50e-2893f25c23a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:25.635497 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:25.667668 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 960/1932] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:51:25 2022] POST /identity/v3/users => generated 293 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:51:25.671734 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d34eb5-cc8a-4289-b69e-c19150b51135 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:25.684267 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d34eb5-cc8a-4289-b69e-c19150b51135 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:25.708772 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 973/1933] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:51:25 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:51:25.729793 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf22172-9d7a-42c6-a50e-2893f25c23a1 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:25.731226 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-3bf22172-9d7a-42c6-a50e-2893f25c23a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:25.775302 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b5456a-1819-43cd-b8ea-29e2d09afa25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:25.776958 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b5456a-1819-43cd-b8ea-29e2d09afa25 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:25.777345 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b5456a-1819-43cd-b8ea-29e2d09afa25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:25.777714 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b5456a-1819-43cd-b8ea-29e2d09afa25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:25.779182 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b5456a-1819-43cd-b8ea-29e2d09afa25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:25.781323 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b5456a-1819-43cd-b8ea-29e2d09afa25 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:25.797350 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 961/1934] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:51:25 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:51:25.801705 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b5456a-1819-43cd-b8ea-29e2d09afa25 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:25.812411 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b5456a-1819-43cd-b8ea-29e2d09afa25 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:25.812619 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf22172-9d7a-42c6-a50e-2893f25c23a1 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:25.816388 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24448c55-f975-4e95-a93b-46a4af8fb086 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:25.817305 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24448c55-f975-4e95-a93b-46a4af8fb086 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:25.817305 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24448c55-f975-4e95-a93b-46a4af8fb086 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:25.817533 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febf1158-02a4-42d8-aff9-c425df290bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:25.817647 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24448c55-f975-4e95-a93b-46a4af8fb086 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:25.818222 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febf1158-02a4-42d8-aff9-c425df290bc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:25.818508 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febf1158-02a4-42d8-aff9-c425df290bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:25.818782 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febf1158-02a4-42d8-aff9-c425df290bc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:25.819553 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febf1158-02a4-42d8-aff9-c425df290bc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:25.821566 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febf1158-02a4-42d8-aff9-c425df290bc1 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:25.829915 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24448c55-f975-4e95-a93b-46a4af8fb086 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:25.831700 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24448c55-f975-4e95-a93b-46a4af8fb086 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:25.833870 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 962/1935] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:51:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:51:25.840081 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febf1158-02a4-42d8-aff9-c425df290bc1 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:25.846575 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 974/1936] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:51:25 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:51:25.851655 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febf1158-02a4-42d8-aff9-c425df290bc1 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:25.859799 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b03e26f-86db-4936-b123-1480ce468bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:25.860887 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b03e26f-86db-4936-b123-1480ce468bfc admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:25.861263 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b03e26f-86db-4936-b123-1480ce468bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:25.861611 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b03e26f-86db-4936-b123-1480ce468bfc admin admin] PATH_INFO: `/v3/projects/69ba2b1537c24d51b726164a1f6b0a05/users/76cee99357614bdeb4b4377161dcc674/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:25.883225 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:25.888681 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b03e26f-86db-4936-b123-1480ce468bfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=69ba2b1537c24d51b726164a1f6b0a05, user_id=76cee99357614bdeb4b4377161dcc674, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:25.895982 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b03e26f-86db-4936-b123-1480ce468bfc admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:26.012724 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24448c55-f975-4e95-a93b-46a4af8fb086 service cinder] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:26.024534 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24448c55-f975-4e95-a93b-46a4af8fb086 service cinder] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:26.035102 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c7604-872a-4f50-83b8-e5cd1761874a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:26.035993 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c7604-872a-4f50-83b8-e5cd1761874a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:26.036314 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c7604-872a-4f50-83b8-e5cd1761874a admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:26.036596 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c7604-872a-4f50-83b8-e5cd1761874a admin admin] PATH_INFO: `/v3/users/82f94ca84f1c42b180ad65dab0c100e8` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:26.037949 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7c7604-872a-4f50-83b8-e5cd1761874a admin admin] RBAC: Authorizing `identity:delete_user(user_id=82f94ca84f1c42b180ad65dab0c100e8)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:26.040356 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7c7604-872a-4f50-83b8-e5cd1761874a admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:26.050190 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 963/1937] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:51:25 2022] PUT /identity/v3/projects/69ba2b1537c24d51b726164a1f6b0a05/users/76cee99357614bdeb4b4377161dcc674/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:51:26.062385 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f8e45f-ed60-4735-a346-b9c04dd3b045 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:26.062739 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f8e45f-ed60-4735-a346-b9c04dd3b045 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:26.063228 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f8e45f-ed60-4735-a346-b9c04dd3b045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:26.094022 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-81f8e45f-ed60-4735-a346-b9c04dd3b045 None None] MFA Rules not processed for user `76cee99357614bdeb4b4377161dcc674`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:26.095242 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-81f8e45f-ed60-4735-a346-b9c04dd3b045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:26.195196 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 964/1938] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:51:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:26.213923 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f8e45f-ed60-4735-a346-b9c04dd3b045 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:26.270082 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ba7c7604-872a-4f50-83b8-e5cd1761874a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82f94ca84f1c42b180ad65dab0c100e8'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:26.271452 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ba7c7604-872a-4f50-83b8-e5cd1761874a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82f94ca84f1c42b180ad65dab0c100e8'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:26.272878 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ba7c7604-872a-4f50-83b8-e5cd1761874a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82f94ca84f1c42b180ad65dab0c100e8'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:26.300939 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f8e45f-ed60-4735-a346-b9c04dd3b045 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:26.304730 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15991ee-17c1-439d-a9c6-87e0672360fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:26.305425 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15991ee-17c1-439d-a9c6-87e0672360fc service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:26.305735 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15991ee-17c1-439d-a9c6-87e0672360fc service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:26.306192 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15991ee-17c1-439d-a9c6-87e0672360fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:26.309251 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-f15991ee-17c1-439d-a9c6-87e0672360fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:26.323814 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15991ee-17c1-439d-a9c6-87e0672360fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:26.325687 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15991ee-17c1-439d-a9c6-87e0672360fc service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:26.338922 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 965/1939] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:51:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:51:26.376802 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ba7c7604-872a-4f50-83b8-e5cd1761874a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82f94ca84f1c42b180ad65dab0c100e8'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:26.404887 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ba7c7604-872a-4f50-83b8-e5cd1761874a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82f94ca84f1c42b180ad65dab0c100e8'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:26.443569 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 975/1940] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:51:26 2022] DELETE /identity/v3/users/82f94ca84f1c42b180ad65dab0c100e8 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:51:26.446763 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c27fa6-bf8c-4e38-bb91-3bec1130da4d None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:26.448209 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c27fa6-bf8c-4e38-bb91-3bec1130da4d None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:26.449054 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c27fa6-bf8c-4e38-bb91-3bec1130da4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:26.482964 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-59c27fa6-bf8c-4e38-bb91-3bec1130da4d None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:26.485980 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-59c27fa6-bf8c-4e38-bb91-3bec1130da4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:26.606395 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 966/1941] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:51:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:26.629312 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c7604-872a-4f50-83b8-e5cd1761874a admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:26.632998 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-ba7c7604-872a-4f50-83b8-e5cd1761874a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:26.669439 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:26.748414 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c7604-872a-4f50-83b8-e5cd1761874a admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:26.765060 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5a5b9f-4958-4a68-9d83-f37f5f6f955f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:26.766156 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5a5b9f-4958-4a68-9d83-f37f5f6f955f service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:26.766641 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5a5b9f-4958-4a68-9d83-f37f5f6f955f service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:26.767130 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5a5b9f-4958-4a68-9d83-f37f5f6f955f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:26.790185 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5a5b9f-4958-4a68-9d83-f37f5f6f955f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:26.791910 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5a5b9f-4958-4a68-9d83-f37f5f6f955f service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:26.804797 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 976/1942] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:51:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:51:27.261643 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c27fa6-bf8c-4e38-bb91-3bec1130da4d None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:27.263572 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-59c27fa6-bf8c-4e38-bb91-3bec1130da4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:27.278669 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c27fa6-bf8c-4e38-bb91-3bec1130da4d None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:27.285542 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6bd659-3441-4f55-8dbd-38592cdf595e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:27.288197 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6bd659-3441-4f55-8dbd-38592cdf595e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:27.288197 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6bd659-3441-4f55-8dbd-38592cdf595e admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:27.288197 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6bd659-3441-4f55-8dbd-38592cdf595e admin admin] PATH_INFO: `/v3/projects/06feb98470404db682c8331ae91c844e` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:27.289461 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6bd659-3441-4f55-8dbd-38592cdf595e admin admin] RBAC: Authorizing `identity:delete_project(project_id=06feb98470404db682c8331ae91c844e)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:27.291944 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6bd659-3441-4f55-8dbd-38592cdf595e admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:27.316771 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:27.416778 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-0f6bd659-3441-4f55-8dbd-38592cdf595e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06feb98470404db682c8331ae91c844e', 'request_id': 'req-0f6bd659-3441-4f55-8dbd-38592cdf595e'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:27.482957 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-0f6bd659-3441-4f55-8dbd-38592cdf595e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06feb98470404db682c8331ae91c844e', 'request_id': 'req-0f6bd659-3441-4f55-8dbd-38592cdf595e'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:27.489259 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-0f6bd659-3441-4f55-8dbd-38592cdf595e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06feb98470404db682c8331ae91c844e', 'request_id': 'req-0f6bd659-3441-4f55-8dbd-38592cdf595e'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:27.501211 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-0f6bd659-3441-4f55-8dbd-38592cdf595e admin admin] The token cache is being invalidate because project 06feb98470404db682c8331ae91c844e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:51:27.501538 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-0f6bd659-3441-4f55-8dbd-38592cdf595e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:27.502815 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-0f6bd659-3441-4f55-8dbd-38592cdf595e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:27.504425 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 967/1943] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:51:27 2022] DELETE /identity/v3/projects/06feb98470404db682c8331ae91c844e => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:27.508694 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6bd659-3441-4f55-8dbd-38592cdf595e admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:27.508829 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5a5b9f-4958-4a68-9d83-f37f5f6f955f service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:27.510127 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-0f6bd659-3441-4f55-8dbd-38592cdf595e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:27.510246 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-5f5a5b9f-4958-4a68-9d83-f37f5f6f955f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:27.526639 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5a5b9f-4958-4a68-9d83-f37f5f6f955f service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:27.596791 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6bd659-3441-4f55-8dbd-38592cdf595e admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:27.600301 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d9b3ce-5cf7-4f0d-9edb-8430192c0ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:27.601836 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d9b3ce-5cf7-4f0d-9edb-8430192c0ddd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:27.602114 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d9b3ce-5cf7-4f0d-9edb-8430192c0ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:27.602553 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d9b3ce-5cf7-4f0d-9edb-8430192c0ddd admin admin] PATH_INFO: `/v3/users/162a6e0a087749b1a523a761c686176e` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:27.607493 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d9b3ce-5cf7-4f0d-9edb-8430192c0ddd admin admin] RBAC: Authorizing `identity:delete_user(user_id=162a6e0a087749b1a523a761c686176e)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:27.607644 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673a6d0c-5ca2-4474-a338-a7a30a3d8e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:27.607745 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d9b3ce-5cf7-4f0d-9edb-8430192c0ddd admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:27.607873 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673a6d0c-5ca2-4474-a338-a7a30a3d8e93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:27.607873 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673a6d0c-5ca2-4474-a338-a7a30a3d8e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:27.607873 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673a6d0c-5ca2-4474-a338-a7a30a3d8e93 admin admin] PATH_INFO: `/v3/users/75f6a65d4f48496bb6c65704beb8bf86` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:27.608988 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673a6d0c-5ca2-4474-a338-a7a30a3d8e93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75f6a65d4f48496bb6c65704beb8bf86)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:27.611014 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673a6d0c-5ca2-4474-a338-a7a30a3d8e93 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:27.911294 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-673a6d0c-5ca2-4474-a338-a7a30a3d8e93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75f6a65d4f48496bb6c65704beb8bf86'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:27.911964 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-673a6d0c-5ca2-4474-a338-a7a30a3d8e93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75f6a65d4f48496bb6c65704beb8bf86'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:27.913095 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-61d9b3ce-5cf7-4f0d-9edb-8430192c0ddd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '162a6e0a087749b1a523a761c686176e'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:27.915126 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-61d9b3ce-5cf7-4f0d-9edb-8430192c0ddd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '162a6e0a087749b1a523a761c686176e'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:27.915906 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-61d9b3ce-5cf7-4f0d-9edb-8430192c0ddd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '162a6e0a087749b1a523a761c686176e'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:27.920290 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-673a6d0c-5ca2-4474-a338-a7a30a3d8e93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75f6a65d4f48496bb6c65704beb8bf86'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:27.921136 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-673a6d0c-5ca2-4474-a338-a7a30a3d8e93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75f6a65d4f48496bb6c65704beb8bf86'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:28.002648 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-61d9b3ce-5cf7-4f0d-9edb-8430192c0ddd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '162a6e0a087749b1a523a761c686176e'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:28.018585 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-61d9b3ce-5cf7-4f0d-9edb-8430192c0ddd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '162a6e0a087749b1a523a761c686176e'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:28.035232 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 977/1944] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:51:27 2022] DELETE /identity/v3/users/162a6e0a087749b1a523a761c686176e => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:28.045152 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9ef8c2-fd4a-4277-a92c-8be681e4f0ae None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:28.045152 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9ef8c2-fd4a-4277-a92c-8be681e4f0ae None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:28.045152 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9ef8c2-fd4a-4277-a92c-8be681e4f0ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:28.061872 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-7d9ef8c2-fd4a-4277-a92c-8be681e4f0ae None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:28.063150 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-7d9ef8c2-fd4a-4277-a92c-8be681e4f0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:28.099981 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-673a6d0c-5ca2-4474-a338-a7a30a3d8e93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75f6a65d4f48496bb6c65704beb8bf86'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:28.103456 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:28.119030 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 968/1945] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:51:27 2022] DELETE /identity/v3/users/75f6a65d4f48496bb6c65704beb8bf86 => generated 0 bytes in 611 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:51:28.130491 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 978/1946] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:51:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:28.142754 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673a6d0c-5ca2-4474-a338-a7a30a3d8e93 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:28.144323 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-673a6d0c-5ca2-4474-a338-a7a30a3d8e93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:28.234888 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:28.239069 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673a6d0c-5ca2-4474-a338-a7a30a3d8e93 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:28.243126 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7897cd6-13c3-4a48-898b-3257d8e2c05a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:28.243841 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7897cd6-13c3-4a48-898b-3257d8e2c05a service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:28.244153 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7897cd6-13c3-4a48-898b-3257d8e2c05a service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:28.244432 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7897cd6-13c3-4a48-898b-3257d8e2c05a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:28.340374 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7897cd6-13c3-4a48-898b-3257d8e2c05a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:28.341921 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7897cd6-13c3-4a48-898b-3257d8e2c05a service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:28.352528 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 969/1947] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:51:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:51:28.747268 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9ef8c2-fd4a-4277-a92c-8be681e4f0ae None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:28.748758 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-7d9ef8c2-fd4a-4277-a92c-8be681e4f0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:28.764495 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7897cd6-13c3-4a48-898b-3257d8e2c05a service neutron] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:28.765914 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-d7897cd6-13c3-4a48-898b-3257d8e2c05a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:28.767918 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9ef8c2-fd4a-4277-a92c-8be681e4f0ae None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:28.772929 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc5c647-b452-46ae-93bf-1c5ca6f7c277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:28.774133 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc5c647-b452-46ae-93bf-1c5ca6f7c277 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:28.774596 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc5c647-b452-46ae-93bf-1c5ca6f7c277 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:28.774886 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc5c647-b452-46ae-93bf-1c5ca6f7c277 admin admin] PATH_INFO: `/v3/users/be0172fefaab46e79f9d65f24cecefbf` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:28.776182 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc5c647-b452-46ae-93bf-1c5ca6f7c277 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be0172fefaab46e79f9d65f24cecefbf)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:28.778684 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc5c647-b452-46ae-93bf-1c5ca6f7c277 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:28.787353 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7897cd6-13c3-4a48-898b-3257d8e2c05a service neutron] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:28.795276 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163c1886-386b-42e0-8fcf-255eb0cfe193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:28.796609 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163c1886-386b-42e0-8fcf-255eb0cfe193 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:28.797136 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163c1886-386b-42e0-8fcf-255eb0cfe193 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:28.797597 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163c1886-386b-42e0-8fcf-255eb0cfe193 admin admin] PATH_INFO: `/v3/projects/732fafa3f71f42ea88fbdcd991bcc661` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:28.799247 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163c1886-386b-42e0-8fcf-255eb0cfe193 admin admin] RBAC: Authorizing `identity:delete_project(project_id=732fafa3f71f42ea88fbdcd991bcc661)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:28.801594 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163c1886-386b-42e0-8fcf-255eb0cfe193 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:29.085438 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-163c1886-386b-42e0-8fcf-255eb0cfe193 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '732fafa3f71f42ea88fbdcd991bcc661', 'request_id': 'req-163c1886-386b-42e0-8fcf-255eb0cfe193'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:29.086550 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-dcc5c647-b452-46ae-93bf-1c5ca6f7c277 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be0172fefaab46e79f9d65f24cecefbf'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:29.087451 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-dcc5c647-b452-46ae-93bf-1c5ca6f7c277 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be0172fefaab46e79f9d65f24cecefbf'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:29.088008 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-dcc5c647-b452-46ae-93bf-1c5ca6f7c277 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be0172fefaab46e79f9d65f24cecefbf'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:29.168439 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-163c1886-386b-42e0-8fcf-255eb0cfe193 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '732fafa3f71f42ea88fbdcd991bcc661', 'request_id': 'req-163c1886-386b-42e0-8fcf-255eb0cfe193'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:29.177893 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-163c1886-386b-42e0-8fcf-255eb0cfe193 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '732fafa3f71f42ea88fbdcd991bcc661', 'request_id': 'req-163c1886-386b-42e0-8fcf-255eb0cfe193'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:29.186664 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-163c1886-386b-42e0-8fcf-255eb0cfe193 admin admin] The token cache is being invalidate because project 732fafa3f71f42ea88fbdcd991bcc661 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:51:29.186949 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-163c1886-386b-42e0-8fcf-255eb0cfe193 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:29.187596 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-163c1886-386b-42e0-8fcf-255eb0cfe193 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:29.189164 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 970/1948] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:51:28 2022] DELETE /identity/v3/projects/732fafa3f71f42ea88fbdcd991bcc661 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:29.190045 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163c1886-386b-42e0-8fcf-255eb0cfe193 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:29.191281 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-163c1886-386b-42e0-8fcf-255eb0cfe193 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:29.231053 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-dcc5c647-b452-46ae-93bf-1c5ca6f7c277 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be0172fefaab46e79f9d65f24cecefbf'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:29.246458 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-dcc5c647-b452-46ae-93bf-1c5ca6f7c277 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be0172fefaab46e79f9d65f24cecefbf'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:29.259472 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 979/1949] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:51:28 2022] DELETE /identity/v3/users/be0172fefaab46e79f9d65f24cecefbf => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:29.260955 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920df101-f1b2-426a-ad7a-f9038e6a4dbf None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:29.261497 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920df101-f1b2-426a-ad7a-f9038e6a4dbf None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:29.261771 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920df101-f1b2-426a-ad7a-f9038e6a4dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:29.262194 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163c1886-386b-42e0-8fcf-255eb0cfe193 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:29.279343 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-920df101-f1b2-426a-ad7a-f9038e6a4dbf None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:29.280577 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-920df101-f1b2-426a-ad7a-f9038e6a4dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:29.318634 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d9c216-1084-4a8a-be93-6d636016ac7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:29.320276 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9c216-1084-4a8a-be93-6d636016ac7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:29.320570 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9c216-1084-4a8a-be93-6d636016ac7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:29.320847 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9c216-1084-4a8a-be93-6d636016ac7e admin admin] PATH_INFO: `/v3/projects/524b80e317bd4cb1a8f7002f82ba8693` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:29.322139 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d9c216-1084-4a8a-be93-6d636016ac7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=524b80e317bd4cb1a8f7002f82ba8693)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:29.324065 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d9c216-1084-4a8a-be93-6d636016ac7e admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:29.344102 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 980/1950] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:51:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:29.345474 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed20a2b-ebb5-4b6c-bc01-af8b9e481bba None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:29.345764 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed20a2b-ebb5-4b6c-bc01-af8b9e481bba None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:29.346041 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed20a2b-ebb5-4b6c-bc01-af8b9e481bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:29.364909 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-aed20a2b-ebb5-4b6c-bc01-af8b9e481bba None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:29.366112 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-aed20a2b-ebb5-4b6c-bc01-af8b9e481bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:29.374764 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 981/1951] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:51:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:29.375974 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43abe3a8-cc63-425c-92b8-4918a7310467 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:29.376304 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43abe3a8-cc63-425c-92b8-4918a7310467 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:29.376580 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43abe3a8-cc63-425c-92b8-4918a7310467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:29.395245 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-43abe3a8-cc63-425c-92b8-4918a7310467 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:29.396502 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-43abe3a8-cc63-425c-92b8-4918a7310467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:29.404878 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 982/1952] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:51:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:29.405654 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43abe3a8-cc63-425c-92b8-4918a7310467 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:29.407187 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-43abe3a8-cc63-425c-92b8-4918a7310467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:29.467446 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43abe3a8-cc63-425c-92b8-4918a7310467 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:29.471319 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e2d28b-6057-4ff0-b823-6d76294e1596 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:29.471991 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e2d28b-6057-4ff0-b823-6d76294e1596 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:29.472309 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e2d28b-6057-4ff0-b823-6d76294e1596 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:29.472586 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e2d28b-6057-4ff0-b823-6d76294e1596 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:29.483385 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e2d28b-6057-4ff0-b823-6d76294e1596 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:29.485641 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e2d28b-6057-4ff0-b823-6d76294e1596 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:29.499774 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 983/1953] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:51:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:51:29.500448 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e2d28b-6057-4ff0-b823-6d76294e1596 service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:29.510707 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e2d28b-6057-4ff0-b823-6d76294e1596 service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:29.536905 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-31d9c216-1084-4a8a-be93-6d636016ac7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '524b80e317bd4cb1a8f7002f82ba8693', 'request_id': 'req-31d9c216-1084-4a8a-be93-6d636016ac7e'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:29.593798 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea434d18-3acb-4664-bb0d-237635abc168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:29.594679 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea434d18-3acb-4664-bb0d-237635abc168 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:29.595019 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea434d18-3acb-4664-bb0d-237635abc168 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:29.595288 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea434d18-3acb-4664-bb0d-237635abc168 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:29.596151 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea434d18-3acb-4664-bb0d-237635abc168 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:51:29.596467 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea434d18-3acb-4664-bb0d-237635abc168 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:29.598870 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea434d18-3acb-4664-bb0d-237635abc168 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:29.615067 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 984/1954] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jun 15 14:51:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:51:29.620420 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea434d18-3acb-4664-bb0d-237635abc168 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:29.648279 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea434d18-3acb-4664-bb0d-237635abc168 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:29.648988 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-31d9c216-1084-4a8a-be93-6d636016ac7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '524b80e317bd4cb1a8f7002f82ba8693', 'request_id': 'req-31d9c216-1084-4a8a-be93-6d636016ac7e'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:29.655490 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-31d9c216-1084-4a8a-be93-6d636016ac7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '524b80e317bd4cb1a8f7002f82ba8693', 'request_id': 'req-31d9c216-1084-4a8a-be93-6d636016ac7e'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:29.667637 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ca48fb-5eb3-4f01-8eba-97e4c55a5fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:29.668354 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ca48fb-5eb3-4f01-8eba-97e4c55a5fb3 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:29.668551 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-31d9c216-1084-4a8a-be93-6d636016ac7e admin admin] The token cache is being invalidate because project 524b80e317bd4cb1a8f7002f82ba8693 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:51:29.668680 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ca48fb-5eb3-4f01-8eba-97e4c55a5fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:29.668842 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-31d9c216-1084-4a8a-be93-6d636016ac7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:29.668986 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ca48fb-5eb3-4f01-8eba-97e4c55a5fb3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:29.669527 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-31d9c216-1084-4a8a-be93-6d636016ac7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:29.669652 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ca48fb-5eb3-4f01-8eba-97e4c55a5fb3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:29.671615 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 971/1955] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:51:29 2022] DELETE /identity/v3/projects/524b80e317bd4cb1a8f7002f82ba8693 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:29.672852 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ca48fb-5eb3-4f01-8eba-97e4c55a5fb3 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:29.675114 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d9c216-1084-4a8a-be93-6d636016ac7e admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:29.676562 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-31d9c216-1084-4a8a-be93-6d636016ac7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:29.682872 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:29.699074 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d9c216-1084-4a8a-be93-6d636016ac7e admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:29.705143 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccb50ee-f4e8-4357-b441-6a045b4dcd92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:29.706128 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb50ee-f4e8-4357-b441-6a045b4dcd92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:29.706482 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb50ee-f4e8-4357-b441-6a045b4dcd92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:29.706829 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb50ee-f4e8-4357-b441-6a045b4dcd92 admin admin] PATH_INFO: `/v3/users/d2d3d4cf21934777b765aa25109ca858` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:29.708299 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccb50ee-f4e8-4357-b441-6a045b4dcd92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2d3d4cf21934777b765aa25109ca858)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:29.713422 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccb50ee-f4e8-4357-b441-6a045b4dcd92 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:29.787774 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 985/1956] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:51:29 2022] POST /identity/v3/projects => generated 373 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:51:29.793799 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ca48fb-5eb3-4f01-8eba-97e4c55a5fb3 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:29.797038 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-62ca48fb-5eb3-4f01-8eba-97e4c55a5fb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:29.876754 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:29.887241 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ca48fb-5eb3-4f01-8eba-97e4c55a5fb3 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:29.893065 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a045f57d-c90c-4732-8ac2-c6839e008acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:29.893808 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a045f57d-c90c-4732-8ac2-c6839e008acc admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:29.894097 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a045f57d-c90c-4732-8ac2-c6839e008acc admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:29.894402 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a045f57d-c90c-4732-8ac2-c6839e008acc admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:29.895158 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a045f57d-c90c-4732-8ac2-c6839e008acc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:29.898108 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a045f57d-c90c-4732-8ac2-c6839e008acc admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:30.024758 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-cccb50ee-f4e8-4357-b441-6a045b4dcd92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2d3d4cf21934777b765aa25109ca858'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:30.025419 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-cccb50ee-f4e8-4357-b441-6a045b4dcd92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2d3d4cf21934777b765aa25109ca858'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:30.032599 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-cccb50ee-f4e8-4357-b441-6a045b4dcd92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2d3d4cf21934777b765aa25109ca858'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:30.033288 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-cccb50ee-f4e8-4357-b441-6a045b4dcd92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2d3d4cf21934777b765aa25109ca858'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:30.076587 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 986/1957] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:51:29 2022] POST /identity/v3/users => generated 287 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:51:30.078050 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a045f57d-c90c-4732-8ac2-c6839e008acc admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:30.079848 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a045f57d-c90c-4732-8ac2-c6839e008acc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:30.095201 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a045f57d-c90c-4732-8ac2-c6839e008acc admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:30.103278 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403d88e6-d0c1-4f58-ab71-d70599c4104c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:30.104406 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403d88e6-d0c1-4f58-ab71-d70599c4104c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:30.104840 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403d88e6-d0c1-4f58-ab71-d70599c4104c admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:30.105368 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403d88e6-d0c1-4f58-ab71-d70599c4104c admin admin] PATH_INFO: `/v3/projects/3ace515628524962a9adad597c499da2` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:30.107254 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403d88e6-d0c1-4f58-ab71-d70599c4104c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ace515628524962a9adad597c499da2)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:30.110832 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403d88e6-d0c1-4f58-ab71-d70599c4104c admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:30.173879 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-cccb50ee-f4e8-4357-b441-6a045b4dcd92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2d3d4cf21934777b765aa25109ca858'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:30.198205 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 972/1958] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:51:29 2022] DELETE /identity/v3/users/d2d3d4cf21934777b765aa25109ca858 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:51:30.199704 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccb50ee-f4e8-4357-b441-6a045b4dcd92 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:30.214896 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-cccb50ee-f4e8-4357-b441-6a045b4dcd92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:30.303466 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccb50ee-f4e8-4357-b441-6a045b4dcd92 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:30.317776 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-403d88e6-d0c1-4f58-ab71-d70599c4104c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ace515628524962a9adad597c499da2', 'request_id': 'req-403d88e6-d0c1-4f58-ab71-d70599c4104c'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:30.346787 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-403d88e6-d0c1-4f58-ab71-d70599c4104c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ace515628524962a9adad597c499da2', 'request_id': 'req-403d88e6-d0c1-4f58-ab71-d70599c4104c'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:30.380855 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c02a82-f736-4c88-995f-abedd90e32a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:30.382207 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c02a82-f736-4c88-995f-abedd90e32a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:30.382714 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c02a82-f736-4c88-995f-abedd90e32a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:30.383191 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c02a82-f736-4c88-995f-abedd90e32a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:30.384189 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c02a82-f736-4c88-995f-abedd90e32a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:30.386031 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c02a82-f736-4c88-995f-abedd90e32a9 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:30.399295 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 973/1959] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:51:30 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:51:30.416844 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c02a82-f736-4c88-995f-abedd90e32a9 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:30.431922 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c02a82-f736-4c88-995f-abedd90e32a9 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:30.436319 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4d2fc8-36e8-4f6c-910f-057314213b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:30.437060 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d2fc8-36e8-4f6c-910f-057314213b08 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:30.437352 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d2fc8-36e8-4f6c-910f-057314213b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:30.437630 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d2fc8-36e8-4f6c-910f-057314213b08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:30.438357 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4d2fc8-36e8-4f6c-910f-057314213b08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:30.439975 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4d2fc8-36e8-4f6c-910f-057314213b08 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:30.451702 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 974/1960] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:51:30 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:51:30.458000 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4d2fc8-36e8-4f6c-910f-057314213b08 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:30.470600 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4d2fc8-36e8-4f6c-910f-057314213b08 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:30.477593 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ebe6f3-d76a-4323-a9c6-7b8a76ac756a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:30.478806 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ebe6f3-d76a-4323-a9c6-7b8a76ac756a admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:30.479340 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ebe6f3-d76a-4323-a9c6-7b8a76ac756a admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:30.479762 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ebe6f3-d76a-4323-a9c6-7b8a76ac756a admin admin] PATH_INFO: `/v3/projects/3c7302bdd2074855a20fcbffa7811a40/users/661bfb914c8046e3b035d17208ca2ec3/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:30.494249 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-403d88e6-d0c1-4f58-ab71-d70599c4104c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ace515628524962a9adad597c499da2', 'request_id': 'req-403d88e6-d0c1-4f58-ab71-d70599c4104c'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:30.499851 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ebe6f3-d76a-4323-a9c6-7b8a76ac756a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c7302bdd2074855a20fcbffa7811a40, user_id=661bfb914c8046e3b035d17208ca2ec3, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:30.503140 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ebe6f3-d76a-4323-a9c6-7b8a76ac756a admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:30.509092 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-403d88e6-d0c1-4f58-ab71-d70599c4104c admin admin] The token cache is being invalidate because project 3ace515628524962a9adad597c499da2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:51:30.509466 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-403d88e6-d0c1-4f58-ab71-d70599c4104c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:30.510595 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-403d88e6-d0c1-4f58-ab71-d70599c4104c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:30.515406 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 987/1961] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:51:30 2022] DELETE /identity/v3/projects/3ace515628524962a9adad597c499da2 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:30.518485 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403d88e6-d0c1-4f58-ab71-d70599c4104c admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:30.524138 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-403d88e6-d0c1-4f58-ab71-d70599c4104c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:30.541277 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403d88e6-d0c1-4f58-ab71-d70599c4104c admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:30.545534 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810bf7d2-7484-4976-be88-d79891f3c343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:30.550219 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810bf7d2-7484-4976-be88-d79891f3c343 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:30.550219 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810bf7d2-7484-4976-be88-d79891f3c343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:30.550219 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810bf7d2-7484-4976-be88-d79891f3c343 admin admin] PATH_INFO: `/v3/users/da86d49a129945c49e94e40fc93d01f9` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:30.550219 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810bf7d2-7484-4976-be88-d79891f3c343 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da86d49a129945c49e94e40fc93d01f9)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:30.558950 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810bf7d2-7484-4976-be88-d79891f3c343 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:30.605696 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 975/1962] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:51:30 2022] PUT /identity/v3/projects/3c7302bdd2074855a20fcbffa7811a40/users/661bfb914c8046e3b035d17208ca2ec3/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:30.621733 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ac3b52-14d9-4744-b847-90bc6aae4b0d None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:30.622578 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ac3b52-14d9-4744-b847-90bc6aae4b0d None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:30.622974 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ac3b52-14d9-4744-b847-90bc6aae4b0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:30.664162 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-30ac3b52-14d9-4744-b847-90bc6aae4b0d None None] MFA Rules not processed for user `661bfb914c8046e3b035d17208ca2ec3`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:30.665464 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-30ac3b52-14d9-4744-b847-90bc6aae4b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:30.733988 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 976/1963] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:51:30 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:51:30.738270 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ac3b52-14d9-4744-b847-90bc6aae4b0d None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:30.739572 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-30ac3b52-14d9-4744-b847-90bc6aae4b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:30.813062 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ac3b52-14d9-4744-b847-90bc6aae4b0d None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:30.818132 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4bac00-bb05-4107-86cb-4ae000b8841d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:30.818808 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4bac00-bb05-4107-86cb-4ae000b8841d admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:30.819097 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4bac00-bb05-4107-86cb-4ae000b8841d admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:30.819378 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4bac00-bb05-4107-86cb-4ae000b8841d admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:30.820701 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4bac00-bb05-4107-86cb-4ae000b8841d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:30.822949 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4bac00-bb05-4107-86cb-4ae000b8841d admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:30.977311 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-810bf7d2-7484-4976-be88-d79891f3c343 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da86d49a129945c49e94e40fc93d01f9'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:30.978690 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-810bf7d2-7484-4976-be88-d79891f3c343 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da86d49a129945c49e94e40fc93d01f9'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:30.982875 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 977/1964] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:51:30 2022] POST /identity/v3/projects => generated 375 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:51:30.983649 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-810bf7d2-7484-4976-be88-d79891f3c343 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da86d49a129945c49e94e40fc93d01f9'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:30.983747 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4bac00-bb05-4107-86cb-4ae000b8841d admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:30.985822 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-1e4bac00-bb05-4107-86cb-4ae000b8841d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:31.071264 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4bac00-bb05-4107-86cb-4ae000b8841d admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:31.076486 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a639129-0e75-4d32-86e6-d0fe75c42911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:31.077398 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a639129-0e75-4d32-86e6-d0fe75c42911 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:31.077682 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a639129-0e75-4d32-86e6-d0fe75c42911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:31.077960 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a639129-0e75-4d32-86e6-d0fe75c42911 admin admin] PATH_INFO: `/v3/projects/a5d06816bda04b9fa404e94c0526def0` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:31.079216 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a639129-0e75-4d32-86e6-d0fe75c42911 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5d06816bda04b9fa404e94c0526def0)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:31.081263 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a639129-0e75-4d32-86e6-d0fe75c42911 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:31.104693 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-810bf7d2-7484-4976-be88-d79891f3c343 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da86d49a129945c49e94e40fc93d01f9'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:31.120014 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-810bf7d2-7484-4976-be88-d79891f3c343 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da86d49a129945c49e94e40fc93d01f9'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:31.132896 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 988/1965] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:51:30 2022] DELETE /identity/v3/users/da86d49a129945c49e94e40fc93d01f9 => generated 0 bytes in 616 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:31.133679 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810bf7d2-7484-4976-be88-d79891f3c343 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:31.135135 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-810bf7d2-7484-4976-be88-d79891f3c343 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:31.214551 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810bf7d2-7484-4976-be88-d79891f3c343 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:31.219924 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0271ca-9daa-4b6b-b58d-3d3196fa7913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:31.220833 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0271ca-9daa-4b6b-b58d-3d3196fa7913 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:31.221116 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0271ca-9daa-4b6b-b58d-3d3196fa7913 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:31.221390 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0271ca-9daa-4b6b-b58d-3d3196fa7913 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:31.222147 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0271ca-9daa-4b6b-b58d-3d3196fa7913 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:31.224322 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0271ca-9daa-4b6b-b58d-3d3196fa7913 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:31.297769 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-7a639129-0e75-4d32-86e6-d0fe75c42911 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5d06816bda04b9fa404e94c0526def0', 'request_id': 'req-7a639129-0e75-4d32-86e6-d0fe75c42911'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:31.403007 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 989/1966] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:51:31 2022] POST /identity/v3/users => generated 294 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:51:31.409083 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0271ca-9daa-4b6b-b58d-3d3196fa7913 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:31.410517 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-7a639129-0e75-4d32-86e6-d0fe75c42911 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5d06816bda04b9fa404e94c0526def0', 'request_id': 'req-7a639129-0e75-4d32-86e6-d0fe75c42911'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:31.432926 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-7a639129-0e75-4d32-86e6-d0fe75c42911 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5d06816bda04b9fa404e94c0526def0', 'request_id': 'req-7a639129-0e75-4d32-86e6-d0fe75c42911'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:31.454170 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0271ca-9daa-4b6b-b58d-3d3196fa7913 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:31.497270 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-7a639129-0e75-4d32-86e6-d0fe75c42911 admin admin] The token cache is being invalidate because project a5d06816bda04b9fa404e94c0526def0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:51:31.497270 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-7a639129-0e75-4d32-86e6-d0fe75c42911 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:31.497270 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-7a639129-0e75-4d32-86e6-d0fe75c42911 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:31.510684 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 978/1967] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:51:30 2022] DELETE /identity/v3/projects/a5d06816bda04b9fa404e94c0526def0 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:31.514063 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b1b992-f2ab-44db-bc76-cef43c80d8d4 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:31.514275 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b1b992-f2ab-44db-bc76-cef43c80d8d4 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:31.514488 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b1b992-f2ab-44db-bc76-cef43c80d8d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:31.545356 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-83b1b992-f2ab-44db-bc76-cef43c80d8d4 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:31.547318 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-83b1b992-f2ab-44db-bc76-cef43c80d8d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:31.629634 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc3e955-a61c-4245-9e17-ff1dd5e2d53c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:31.630505 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc3e955-a61c-4245-9e17-ff1dd5e2d53c admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:31.630901 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc3e955-a61c-4245-9e17-ff1dd5e2d53c admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:31.632139 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc3e955-a61c-4245-9e17-ff1dd5e2d53c admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:31.632594 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc3e955-a61c-4245-9e17-ff1dd5e2d53c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:31.635507 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc3e955-a61c-4245-9e17-ff1dd5e2d53c admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:31.653730 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 990/1968] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:51:31 2022] GET /identity/v3/roles => generated 1420 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:51:31.659485 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:31.661229 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc3e955-a61c-4245-9e17-ff1dd5e2d53c admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:31.668765 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-5fc3e955-a61c-4245-9e17-ff1dd5e2d53c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:31.685133 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 979/1969] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:51:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:31.693386 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbf1f83-e328-4789-b86b-253aa5b3891a None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:31.694105 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbf1f83-e328-4789-b86b-253aa5b3891a None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:31.695109 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbf1f83-e328-4789-b86b-253aa5b3891a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:31.698651 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc3e955-a61c-4245-9e17-ff1dd5e2d53c admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:31.714016 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47c1284-ba09-4de0-a471-99afd694d79f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:31.715748 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47c1284-ba09-4de0-a471-99afd694d79f admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:31.716180 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47c1284-ba09-4de0-a471-99afd694d79f admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:31.716574 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47c1284-ba09-4de0-a471-99afd694d79f admin admin] PATH_INFO: `/v3/projects/886597d1bd9d45bf8f064e95687fd55c/users/0b2975db7cac44e1a55bfdb91fc3a278/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:31.729155 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-adbf1f83-e328-4789-b86b-253aa5b3891a None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:31.730380 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-adbf1f83-e328-4789-b86b-253aa5b3891a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:31.739290 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 980/1970] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:51:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:31.749951 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbf1f83-e328-4789-b86b-253aa5b3891a None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:31.751892 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47c1284-ba09-4de0-a471-99afd694d79f admin admin] RBAC: Authorizing `identity:create_grant(project_id=886597d1bd9d45bf8f064e95687fd55c, user_id=0b2975db7cac44e1a55bfdb91fc3a278, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:31.757925 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47c1284-ba09-4de0-a471-99afd694d79f admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:31.772308 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbf1f83-e328-4789-b86b-253aa5b3891a None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:31.777627 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c68018d-79ad-4e43-9145-38b38ac98b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:31.778383 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c68018d-79ad-4e43-9145-38b38ac98b5d admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:31.778664 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c68018d-79ad-4e43-9145-38b38ac98b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:31.778939 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c68018d-79ad-4e43-9145-38b38ac98b5d admin admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:31.779804 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c68018d-79ad-4e43-9145-38b38ac98b5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:51:31.780146 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c68018d-79ad-4e43-9145-38b38ac98b5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:31.781764 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c68018d-79ad-4e43-9145-38b38ac98b5d admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:31.801636 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 981/1971] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jun 15 14:51:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:51:31.806904 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c68018d-79ad-4e43-9145-38b38ac98b5d admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:31.819839 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c68018d-79ad-4e43-9145-38b38ac98b5d admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:31.825200 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b9c7de-8d53-43c4-8376-67c1a3132879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:31.825874 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b9c7de-8d53-43c4-8376-67c1a3132879 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:31.826164 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b9c7de-8d53-43c4-8376-67c1a3132879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:31.826447 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b9c7de-8d53-43c4-8376-67c1a3132879 admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:31.827158 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b9c7de-8d53-43c4-8376-67c1a3132879 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:31.829458 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b9c7de-8d53-43c4-8376-67c1a3132879 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:31.884112 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:31.891339 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 991/1972] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:51:31 2022] PUT /identity/v3/projects/886597d1bd9d45bf8f064e95687fd55c/users/0b2975db7cac44e1a55bfdb91fc3a278/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:31.894316 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47c1284-ba09-4de0-a471-99afd694d79f admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:31.896389 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-f47c1284-ba09-4de0-a471-99afd694d79f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:31.964920 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 982/1973] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:51:31 2022] POST /identity/v3/projects => generated 359 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:51:31.964920 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b9c7de-8d53-43c4-8376-67c1a3132879 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:31.993980 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47c1284-ba09-4de0-a471-99afd694d79f admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:32.042126 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b9c7de-8d53-43c4-8376-67c1a3132879 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:32.053138 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b65e4f9-1827-4fbe-99a2-eb52d153453f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:32.054621 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b65e4f9-1827-4fbe-99a2-eb52d153453f admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:32.055211 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b65e4f9-1827-4fbe-99a2-eb52d153453f admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:32.055654 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b65e4f9-1827-4fbe-99a2-eb52d153453f admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:32.056750 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b65e4f9-1827-4fbe-99a2-eb52d153453f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:32.058399 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b65e4f9-1827-4fbe-99a2-eb52d153453f admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:32.072315 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 983/1974] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:51:31 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:51:32.076715 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b65e4f9-1827-4fbe-99a2-eb52d153453f admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:32.080231 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878bbbd6-c74d-4607-8bba-f9d393d46c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:32.080231 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878bbbd6-c74d-4607-8bba-f9d393d46c5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:32.083301 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878bbbd6-c74d-4607-8bba-f9d393d46c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:32.083301 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878bbbd6-c74d-4607-8bba-f9d393d46c5f admin admin] PATH_INFO: `/v3/projects/6cd6878b37194d2fa2116be9fe9da3c7` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:32.085122 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878bbbd6-c74d-4607-8bba-f9d393d46c5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cd6878b37194d2fa2116be9fe9da3c7)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:32.088347 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878bbbd6-c74d-4607-8bba-f9d393d46c5f admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:32.089264 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b65e4f9-1827-4fbe-99a2-eb52d153453f admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:32.094774 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0526bfa6-9240-433e-b962-98f8cea87dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:32.095564 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0526bfa6-9240-433e-b962-98f8cea87dd8 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:32.095926 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0526bfa6-9240-433e-b962-98f8cea87dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:32.096437 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0526bfa6-9240-433e-b962-98f8cea87dd8 admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:32.097209 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0526bfa6-9240-433e-b962-98f8cea87dd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:32.099202 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0526bfa6-9240-433e-b962-98f8cea87dd8 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:32.313567 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 984/1975] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:51:32 2022] POST /identity/v3/users => generated 280 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:51:32.314875 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0526bfa6-9240-433e-b962-98f8cea87dd8 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:32.326626 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0526bfa6-9240-433e-b962-98f8cea87dd8 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:32.331934 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71891343-cafb-4675-878b-91799d1dd606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:32.332955 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71891343-cafb-4675-878b-91799d1dd606 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:32.333262 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71891343-cafb-4675-878b-91799d1dd606 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:32.333586 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71891343-cafb-4675-878b-91799d1dd606 admin admin] PATH_INFO: `/v3/projects/886597d1bd9d45bf8f064e95687fd55c/users/0b2975db7cac44e1a55bfdb91fc3a278/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:32.337928 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71891343-cafb-4675-878b-91799d1dd606 admin admin] RBAC: Authorizing `identity:create_grant(project_id=886597d1bd9d45bf8f064e95687fd55c, user_id=0b2975db7cac44e1a55bfdb91fc3a278, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:32.339410 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71891343-cafb-4675-878b-91799d1dd606 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:32.354637 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 985/1976] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:51:32 2022] PUT /identity/v3/projects/886597d1bd9d45bf8f064e95687fd55c/users/0b2975db7cac44e1a55bfdb91fc3a278/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:32.355931 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71891343-cafb-4675-878b-91799d1dd606 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:32.389554 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-878bbbd6-c74d-4607-8bba-f9d393d46c5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cd6878b37194d2fa2116be9fe9da3c7', 'request_id': 'req-878bbbd6-c74d-4607-8bba-f9d393d46c5f'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:32.397062 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-878bbbd6-c74d-4607-8bba-f9d393d46c5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cd6878b37194d2fa2116be9fe9da3c7', 'request_id': 'req-878bbbd6-c74d-4607-8bba-f9d393d46c5f'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:32.433233 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71891343-cafb-4675-878b-91799d1dd606 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:32.494588 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-878bbbd6-c74d-4607-8bba-f9d393d46c5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cd6878b37194d2fa2116be9fe9da3c7', 'request_id': 'req-878bbbd6-c74d-4607-8bba-f9d393d46c5f'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:32.508714 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-878bbbd6-c74d-4607-8bba-f9d393d46c5f admin admin] The token cache is being invalidate because project 6cd6878b37194d2fa2116be9fe9da3c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:51:32.509375 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-878bbbd6-c74d-4607-8bba-f9d393d46c5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:32.510872 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-878bbbd6-c74d-4607-8bba-f9d393d46c5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:32.511840 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f98a87-8600-4890-b63c-5838a99dc779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:32.513175 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 992/1977] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:51:31 2022] DELETE /identity/v3/projects/6cd6878b37194d2fa2116be9fe9da3c7 => generated 0 bytes in 619 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:51:32.516572 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f98a87-8600-4890-b63c-5838a99dc779 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:32.516572 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f98a87-8600-4890-b63c-5838a99dc779 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:32.516572 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f98a87-8600-4890-b63c-5838a99dc779 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:32.516853 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6925e7-2721-4545-8ae6-4d0f4a203f62 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:32.516853 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6925e7-2721-4545-8ae6-4d0f4a203f62 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:32.516853 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6925e7-2721-4545-8ae6-4d0f4a203f62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:32.517081 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f98a87-8600-4890-b63c-5838a99dc779 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:32.518284 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f98a87-8600-4890-b63c-5838a99dc779 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:32.543004 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 986/1978] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:51:32 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:51:32.545537 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ebbf73-e6e6-44ac-bb98-a07ea65d7e23 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:32.545993 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ebbf73-e6e6-44ac-bb98-a07ea65d7e23 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:32.546401 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ebbf73-e6e6-44ac-bb98-a07ea65d7e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:32.557707 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-be6925e7-2721-4545-8ae6-4d0f4a203f62 None None] MFA Rules not processed for user `0b2975db7cac44e1a55bfdb91fc3a278`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:32.558924 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-be6925e7-2721-4545-8ae6-4d0f4a203f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:32.578602 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-44ebbf73-e6e6-44ac-bb98-a07ea65d7e23 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:32.580627 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-44ebbf73-e6e6-44ac-bb98-a07ea65d7e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:32.593198 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 987/1979] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:51:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:32.594334 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ebbf73-e6e6-44ac-bb98-a07ea65d7e23 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:32.596199 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-44ebbf73-e6e6-44ac-bb98-a07ea65d7e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:32.613031 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ebbf73-e6e6-44ac-bb98-a07ea65d7e23 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:32.619453 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c218730b-17c2-4fda-aa6f-d97094b312f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:32.620175 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c218730b-17c2-4fda-aa6f-d97094b312f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:32.620468 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c218730b-17c2-4fda-aa6f-d97094b312f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:32.620748 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c218730b-17c2-4fda-aa6f-d97094b312f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:32.621727 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c218730b-17c2-4fda-aa6f-d97094b312f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:32.623155 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c218730b-17c2-4fda-aa6f-d97094b312f5 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:32.634483 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 988/1980] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:51:32 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:51:32.636827 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7a9603-13ca-40f7-8297-6b06372aa031 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:32.637236 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7a9603-13ca-40f7-8297-6b06372aa031 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:32.637610 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7a9603-13ca-40f7-8297-6b06372aa031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:32.651891 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 993/1981] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:51:32 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:32.652722 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6925e7-2721-4545-8ae6-4d0f4a203f62 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:32.661043 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-8c7a9603-13ca-40f7-8297-6b06372aa031 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:32.662335 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-8c7a9603-13ca-40f7-8297-6b06372aa031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:32.667845 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6925e7-2721-4545-8ae6-4d0f4a203f62 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:32.673408 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 989/1982] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:51:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:32.674446 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7a9603-13ca-40f7-8297-6b06372aa031 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:32.676082 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-8c7a9603-13ca-40f7-8297-6b06372aa031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:32.676886 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bb8a0c-f3e7-4c76-90e1-f37938bc2573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:32.677910 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb8a0c-f3e7-4c76-90e1-f37938bc2573 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:32.678198 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb8a0c-f3e7-4c76-90e1-f37938bc2573 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:32.678485 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb8a0c-f3e7-4c76-90e1-f37938bc2573 admin admin] PATH_INFO: `/v3/projects/6bb8a428b3aa44f9adbbc9f77e17b870/users/057fbf7294874f1f8f7693291a32b903/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:32.698224 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bb8a0c-f3e7-4c76-90e1-f37938bc2573 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bb8a428b3aa44f9adbbc9f77e17b870, user_id=057fbf7294874f1f8f7693291a32b903, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:32.701271 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bb8a0c-f3e7-4c76-90e1-f37938bc2573 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:32.762434 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7a9603-13ca-40f7-8297-6b06372aa031 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:32.762434 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829dbdaa-05e7-4fe7-8f91-77d78205520d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:32.762434 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829dbdaa-05e7-4fe7-8f91-77d78205520d service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:32.762434 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829dbdaa-05e7-4fe7-8f91-77d78205520d service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:32.762986 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829dbdaa-05e7-4fe7-8f91-77d78205520d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:32.762986 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-829dbdaa-05e7-4fe7-8f91-77d78205520d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:32.821925 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829dbdaa-05e7-4fe7-8f91-77d78205520d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:32.823292 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829dbdaa-05e7-4fe7-8f91-77d78205520d service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:32.834310 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 990/1983] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:51:32 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:51:32.836965 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829dbdaa-05e7-4fe7-8f91-77d78205520d service cinder] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:32.850424 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829dbdaa-05e7-4fe7-8f91-77d78205520d service cinder] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:32.856850 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0b0735-4c04-4303-9159-ec3dbd528367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:32.857622 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0b0735-4c04-4303-9159-ec3dbd528367 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:32.858774 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0b0735-4c04-4303-9159-ec3dbd528367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:32.858774 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0b0735-4c04-4303-9159-ec3dbd528367 admin admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:32.859095 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0b0735-4c04-4303-9159-ec3dbd528367 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:51:32.859264 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0b0735-4c04-4303-9159-ec3dbd528367 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:32.860928 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0b0735-4c04-4303-9159-ec3dbd528367 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:32.880501 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 991/1984] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jun 15 14:51:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:51:32.885930 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 994/1985] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:51:32 2022] PUT /identity/v3/projects/6bb8a428b3aa44f9adbbc9f77e17b870/users/057fbf7294874f1f8f7693291a32b903/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:32.887690 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0b0735-4c04-4303-9159-ec3dbd528367 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:32.898366 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24457632-a95c-4697-b1a8-3f10100efbed None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:32.898694 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24457632-a95c-4697-b1a8-3f10100efbed None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:32.898979 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24457632-a95c-4697-b1a8-3f10100efbed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:32.903971 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0b0735-4c04-4303-9159-ec3dbd528367 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:32.938838 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-24457632-a95c-4697-b1a8-3f10100efbed None None] MFA Rules not processed for user `057fbf7294874f1f8f7693291a32b903`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:32.940066 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-24457632-a95c-4697-b1a8-3f10100efbed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:32.988493 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ccd073-7571-4805-aed9-377fd8a026a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:32.989539 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ccd073-7571-4805-aed9-377fd8a026a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:32.989882 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ccd073-7571-4805-aed9-377fd8a026a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:32.990235 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ccd073-7571-4805-aed9-377fd8a026a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:32.991017 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ccd073-7571-4805-aed9-377fd8a026a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:32.993835 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ccd073-7571-4805-aed9-377fd8a026a8 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:33.030851 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 995/1986] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:51:32 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:51:33.063950 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24457632-a95c-4697-b1a8-3f10100efbed None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:33.065568 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-24457632-a95c-4697-b1a8-3f10100efbed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:33.126207 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24457632-a95c-4697-b1a8-3f10100efbed None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:33.130098 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9787584-28a6-46bf-a39a-623c74ce9296 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:33.130560 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9787584-28a6-46bf-a39a-623c74ce9296 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:33.130841 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9787584-28a6-46bf-a39a-623c74ce9296 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:33.131117 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9787584-28a6-46bf-a39a-623c74ce9296 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:33.141595 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9787584-28a6-46bf-a39a-623c74ce9296 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:33.143049 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9787584-28a6-46bf-a39a-623c74ce9296 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:33.153352 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 996/1987] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:51:33 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:51:33.169553 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9787584-28a6-46bf-a39a-623c74ce9296 service cinder] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:33.182084 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9787584-28a6-46bf-a39a-623c74ce9296 service cinder] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:33.187329 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a88b11f-25e1-41d9-bb4f-ce4061532831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:33.188269 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a88b11f-25e1-41d9-bb4f-ce4061532831 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:33.188558 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a88b11f-25e1-41d9-bb4f-ce4061532831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:33.188838 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a88b11f-25e1-41d9-bb4f-ce4061532831 admin admin] PATH_INFO: `/v3/users/057fbf7294874f1f8f7693291a32b903` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:33.190089 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a88b11f-25e1-41d9-bb4f-ce4061532831 admin admin] RBAC: Authorizing `identity:delete_user(user_id=057fbf7294874f1f8f7693291a32b903)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:33.192026 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a88b11f-25e1-41d9-bb4f-ce4061532831 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:33.230480 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=81,keystone:DELETE=14,keystone:INSERT=5 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:33.281055 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 992/1988] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:51:32 2022] POST /identity/v3/projects => generated 401 bytes in 397 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:51:33.282302 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ccd073-7571-4805-aed9-377fd8a026a8 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:33.361725 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ccd073-7571-4805-aed9-377fd8a026a8 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:33.370166 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1437e8-54fc-4f2e-85e4-fd900a4aa6e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:33.371156 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1437e8-54fc-4f2e-85e4-fd900a4aa6e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:33.371435 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1437e8-54fc-4f2e-85e4-fd900a4aa6e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:33.371860 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1437e8-54fc-4f2e-85e4-fd900a4aa6e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:33.372614 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1437e8-54fc-4f2e-85e4-fd900a4aa6e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:33.375766 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1437e8-54fc-4f2e-85e4-fd900a4aa6e7 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:33.415094 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:33.627041 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 993/1989] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:51:33 2022] POST /identity/v3/users => generated 301 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:51:33.628398 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1437e8-54fc-4f2e-85e4-fd900a4aa6e7 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:33.632473 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-9a88b11f-25e1-41d9-bb4f-ce4061532831 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '057fbf7294874f1f8f7693291a32b903'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:33.634605 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-9a88b11f-25e1-41d9-bb4f-ce4061532831 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '057fbf7294874f1f8f7693291a32b903'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:33.635473 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-9a88b11f-25e1-41d9-bb4f-ce4061532831 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '057fbf7294874f1f8f7693291a32b903'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:33.646386 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1437e8-54fc-4f2e-85e4-fd900a4aa6e7 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:33.653773 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b0d9de-ea34-4bb4-b00d-7cd24f936f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:33.654742 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b0d9de-ea34-4bb4-b00d-7cd24f936f65 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:33.654742 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b0d9de-ea34-4bb4-b00d-7cd24f936f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:33.655181 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b0d9de-ea34-4bb4-b00d-7cd24f936f65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:33.656057 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b0d9de-ea34-4bb4-b00d-7cd24f936f65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:33.658240 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b0d9de-ea34-4bb4-b00d-7cd24f936f65 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:33.676124 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 994/1990] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:51:33 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:51:33.690182 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b0d9de-ea34-4bb4-b00d-7cd24f936f65 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:33.692143 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-40b0d9de-ea34-4bb4-b00d-7cd24f936f65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:33.706231 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b0d9de-ea34-4bb4-b00d-7cd24f936f65 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:33.712310 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bbec4a-5a3d-417d-9e02-96b29347444b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:33.712988 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bbec4a-5a3d-417d-9e02-96b29347444b admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:33.713270 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bbec4a-5a3d-417d-9e02-96b29347444b admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:33.713541 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bbec4a-5a3d-417d-9e02-96b29347444b admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:33.714284 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bbec4a-5a3d-417d-9e02-96b29347444b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:33.715942 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bbec4a-5a3d-417d-9e02-96b29347444b admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:33.727295 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 995/1991] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:51:33 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:51:33.731390 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bbec4a-5a3d-417d-9e02-96b29347444b admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:33.741184 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bbec4a-5a3d-417d-9e02-96b29347444b admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:33.748439 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8178429-7fa9-43ea-805d-e08b92d560fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:33.749356 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8178429-7fa9-43ea-805d-e08b92d560fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:33.749633 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8178429-7fa9-43ea-805d-e08b92d560fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:33.749909 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8178429-7fa9-43ea-805d-e08b92d560fa admin admin] PATH_INFO: `/v3/projects/d72406625c194437852460ba8f03918e/users/9ce606720c304964b46b6786ee829edb/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:33.767488 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8178429-7fa9-43ea-805d-e08b92d560fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=d72406625c194437852460ba8f03918e, user_id=9ce606720c304964b46b6786ee829edb, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:33.770217 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8178429-7fa9-43ea-805d-e08b92d560fa admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:33.825697 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-9a88b11f-25e1-41d9-bb4f-ce4061532831 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '057fbf7294874f1f8f7693291a32b903'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:33.842182 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-9a88b11f-25e1-41d9-bb4f-ce4061532831 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '057fbf7294874f1f8f7693291a32b903'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:33.855666 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 997/1992] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:51:33 2022] DELETE /identity/v3/users/057fbf7294874f1f8f7693291a32b903 => generated 0 bytes in 687 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:51:33.860289 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573bdbbc-1523-4b2a-a9c2-db697022e207 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:33.860587 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573bdbbc-1523-4b2a-a9c2-db697022e207 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:33.860873 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573bdbbc-1523-4b2a-a9c2-db697022e207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:33.877977 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-573bdbbc-1523-4b2a-a9c2-db697022e207 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:33.879105 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-573bdbbc-1523-4b2a-a9c2-db697022e207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:33.942226 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 998/1993] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:51:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:51:33.954009 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573bdbbc-1523-4b2a-a9c2-db697022e207 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:33.955629 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-573bdbbc-1523-4b2a-a9c2-db697022e207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:33.964583 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 996/1994] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:51:33 2022] PUT /identity/v3/projects/d72406625c194437852460ba8f03918e/users/9ce606720c304964b46b6786ee829edb/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:51:33.975781 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbd6770-33bb-426f-8121-0f526f1a5c83 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:33.976429 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbd6770-33bb-426f-8121-0f526f1a5c83 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:33.979149 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbd6770-33bb-426f-8121-0f526f1a5c83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:34.005642 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-2bbd6770-33bb-426f-8121-0f526f1a5c83 None None] MFA Rules not processed for user `9ce606720c304964b46b6786ee829edb`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:34.007577 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-2bbd6770-33bb-426f-8121-0f526f1a5c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:34.021456 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573bdbbc-1523-4b2a-a9c2-db697022e207 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:34.074086 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ac4cfa-bde9-47f2-8324-1dc906666ffe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:34.074852 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ac4cfa-bde9-47f2-8324-1dc906666ffe service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:34.075035 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ac4cfa-bde9-47f2-8324-1dc906666ffe service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:34.075305 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ac4cfa-bde9-47f2-8324-1dc906666ffe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:34.081862 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 997/1995] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:51:33 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:51:34.086750 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbd6770-33bb-426f-8121-0f526f1a5c83 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:34.157297 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ac4cfa-bde9-47f2-8324-1dc906666ffe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:34.158910 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ac4cfa-bde9-47f2-8324-1dc906666ffe service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:34.161891 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbd6770-33bb-426f-8121-0f526f1a5c83 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:34.169279 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223d08ef-d2e0-4d53-984d-32efc2b14aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:34.169980 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223d08ef-d2e0-4d53-984d-32efc2b14aeb admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:34.170254 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223d08ef-d2e0-4d53-984d-32efc2b14aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:34.170533 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223d08ef-d2e0-4d53-984d-32efc2b14aeb admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:34.171458 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223d08ef-d2e0-4d53-984d-32efc2b14aeb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:34.171567 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 999/1996] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:51:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:51:34.173417 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223d08ef-d2e0-4d53-984d-32efc2b14aeb admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:34.440235 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ac4cfa-bde9-47f2-8324-1dc906666ffe service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:34.449151 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 998/1997] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:51:34 2022] POST /identity/v3/projects => generated 399 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:51:34.465800 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ac4cfa-bde9-47f2-8324-1dc906666ffe service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:34.544915 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf00266-03a0-41d5-ab22-ad9106ef6df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:34.565305 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf00266-03a0-41d5-ab22-ad9106ef6df1 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:34.565305 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf00266-03a0-41d5-ab22-ad9106ef6df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:34.565305 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf00266-03a0-41d5-ab22-ad9106ef6df1 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:34.565305 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf00266-03a0-41d5-ab22-ad9106ef6df1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:34.565305 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf00266-03a0-41d5-ab22-ad9106ef6df1 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:34.591959 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:34.720569 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1000/1998] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:51:34 2022] POST /identity/v3/users => generated 306 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:51:34.723991 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223d08ef-d2e0-4d53-984d-32efc2b14aeb admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:34.740470 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223d08ef-d2e0-4d53-984d-32efc2b14aeb admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:34.769241 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390d3fb1-2ae9-47d1-a59a-258493fa0bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:34.769958 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390d3fb1-2ae9-47d1-a59a-258493fa0bb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:34.770265 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390d3fb1-2ae9-47d1-a59a-258493fa0bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:34.771101 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390d3fb1-2ae9-47d1-a59a-258493fa0bb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:34.771319 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390d3fb1-2ae9-47d1-a59a-258493fa0bb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:34.773593 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390d3fb1-2ae9-47d1-a59a-258493fa0bb3 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:34.796432 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 999/1999] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:51:34 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:51:34.798663 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf00266-03a0-41d5-ab22-ad9106ef6df1 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:34.810521 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf00266-03a0-41d5-ab22-ad9106ef6df1 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:34.816608 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a971c1-63f7-468f-86f7-868650f74007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:34.818146 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a971c1-63f7-468f-86f7-868650f74007 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:34.818496 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a971c1-63f7-468f-86f7-868650f74007 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:34.818842 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a971c1-63f7-468f-86f7-868650f74007 admin admin] PATH_INFO: `/v3/projects/2137841a71074dd482a694eaeb4803e3/users/272317017d1e42f5b681dbdd6fe75538/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:34.873305 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a971c1-63f7-468f-86f7-868650f74007 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2137841a71074dd482a694eaeb4803e3, user_id=272317017d1e42f5b681dbdd6fe75538, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:34.873305 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a971c1-63f7-468f-86f7-868650f74007 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:35.022416 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1001/2000] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:51:34 2022] PUT /identity/v3/projects/2137841a71074dd482a694eaeb4803e3/users/272317017d1e42f5b681dbdd6fe75538/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:35.027146 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390d3fb1-2ae9-47d1-a59a-258493fa0bb3 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:35.029074 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-390d3fb1-2ae9-47d1-a59a-258493fa0bb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:35.034263 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a971c1-63f7-468f-86f7-868650f74007 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:35.066908 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390d3fb1-2ae9-47d1-a59a-258493fa0bb3 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:35.135761 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a971c1-63f7-468f-86f7-868650f74007 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:35.142682 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259e5eac-9eaf-4dcc-886d-d7406fee3ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:35.143665 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e5eac-9eaf-4dcc-886d-d7406fee3ab3 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:35.143999 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e5eac-9eaf-4dcc-886d-d7406fee3ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:35.144256 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e5eac-9eaf-4dcc-886d-d7406fee3ab3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:35.144919 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259e5eac-9eaf-4dcc-886d-d7406fee3ab3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:35.146663 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259e5eac-9eaf-4dcc-886d-d7406fee3ab3 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:35.181908 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1002/2001] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:51:35 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:51:35.181908 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259e5eac-9eaf-4dcc-886d-d7406fee3ab3 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:35.182308 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc08306-0d66-4e44-8ec7-e95eefb91d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:35.182308 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc08306-0d66-4e44-8ec7-e95eefb91d62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:35.182308 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc08306-0d66-4e44-8ec7-e95eefb91d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:35.182308 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc08306-0d66-4e44-8ec7-e95eefb91d62 admin admin] PATH_INFO: `/v3/projects/6bb8a428b3aa44f9adbbc9f77e17b870` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:35.183551 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc08306-0d66-4e44-8ec7-e95eefb91d62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bb8a428b3aa44f9adbbc9f77e17b870)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:35.186813 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc08306-0d66-4e44-8ec7-e95eefb91d62 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:35.187094 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259e5eac-9eaf-4dcc-886d-d7406fee3ab3 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:35.193091 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cb31ae-1600-4290-a91b-1f7fe2655bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:35.194991 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cb31ae-1600-4290-a91b-1f7fe2655bdb admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:35.195742 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cb31ae-1600-4290-a91b-1f7fe2655bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:35.196403 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cb31ae-1600-4290-a91b-1f7fe2655bdb admin admin] PATH_INFO: `/v3/projects/2137841a71074dd482a694eaeb4803e3/users/272317017d1e42f5b681dbdd6fe75538/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:35.199505 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cb31ae-1600-4290-a91b-1f7fe2655bdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2137841a71074dd482a694eaeb4803e3, user_id=272317017d1e42f5b681dbdd6fe75538, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:35.202534 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cb31ae-1600-4290-a91b-1f7fe2655bdb admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:35.216443 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1003/2002] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:51:35 2022] PUT /identity/v3/projects/2137841a71074dd482a694eaeb4803e3/users/272317017d1e42f5b681dbdd6fe75538/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:51:35.235054 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1448699b-7571-483b-bc46-ad02a4c082da None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:35.235494 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1448699b-7571-483b-bc46-ad02a4c082da None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:35.235923 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1448699b-7571-483b-bc46-ad02a4c082da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:35.274628 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-1448699b-7571-483b-bc46-ad02a4c082da None None] MFA Rules not processed for user `272317017d1e42f5b681dbdd6fe75538`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:35.276217 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-1448699b-7571-483b-bc46-ad02a4c082da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:35.284706 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:35.335721 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-8dc08306-0d66-4e44-8ec7-e95eefb91d62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bb8a428b3aa44f9adbbc9f77e17b870', 'request_id': 'req-8dc08306-0d66-4e44-8ec7-e95eefb91d62'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:35.375451 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1004/2003] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:51:35 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:51:35.387998 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1448699b-7571-483b-bc46-ad02a4c082da None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:35.389666 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-1448699b-7571-483b-bc46-ad02a4c082da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:35.431744 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-8dc08306-0d66-4e44-8ec7-e95eefb91d62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bb8a428b3aa44f9adbbc9f77e17b870', 'request_id': 'req-8dc08306-0d66-4e44-8ec7-e95eefb91d62'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:35.439625 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-8dc08306-0d66-4e44-8ec7-e95eefb91d62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bb8a428b3aa44f9adbbc9f77e17b870', 'request_id': 'req-8dc08306-0d66-4e44-8ec7-e95eefb91d62'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:35.455464 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-8dc08306-0d66-4e44-8ec7-e95eefb91d62 admin admin] The token cache is being invalidate because project 6bb8a428b3aa44f9adbbc9f77e17b870 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:51:35.456177 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-8dc08306-0d66-4e44-8ec7-e95eefb91d62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:35.456890 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-8dc08306-0d66-4e44-8ec7-e95eefb91d62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:35.461701 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1000/2004] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:51:35 2022] DELETE /identity/v3/projects/6bb8a428b3aa44f9adbbc9f77e17b870 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:35.465280 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1448699b-7571-483b-bc46-ad02a4c082da None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:35.470958 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037fcc50-0f05-4de9-997b-3313eee8a9a4 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:35.471771 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037fcc50-0f05-4de9-997b-3313eee8a9a4 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:35.471929 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037fcc50-0f05-4de9-997b-3313eee8a9a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:35.498090 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-037fcc50-0f05-4de9-997b-3313eee8a9a4 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:35.500381 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-037fcc50-0f05-4de9-997b-3313eee8a9a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:35.575701 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721817dd-e23a-46e2-aea3-ff70b520ecde service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:35.576431 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721817dd-e23a-46e2-aea3-ff70b520ecde service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:35.576761 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721817dd-e23a-46e2-aea3-ff70b520ecde service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:35.577048 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721817dd-e23a-46e2-aea3-ff70b520ecde service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:35.580189 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-721817dd-e23a-46e2-aea3-ff70b520ecde service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:35.622962 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1001/2005] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:51:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:51:35.634763 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cca4002-d2b5-49ed-957c-968a97b64883 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:35.635245 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cca4002-d2b5-49ed-957c-968a97b64883 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:35.635537 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cca4002-d2b5-49ed-957c-968a97b64883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:35.667742 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-3cca4002-d2b5-49ed-957c-968a97b64883 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:35.669740 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-3cca4002-d2b5-49ed-957c-968a97b64883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:35.683171 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721817dd-e23a-46e2-aea3-ff70b520ecde service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:35.684952 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721817dd-e23a-46e2-aea3-ff70b520ecde service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:35.700510 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1005/2006] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:51:35 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:51:35.727494 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1002/2007] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:51:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:35.731850 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721817dd-e23a-46e2-aea3-ff70b520ecde service cinder] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:35.758328 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721817dd-e23a-46e2-aea3-ff70b520ecde service cinder] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:35.765574 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89d2b38-a118-4786-ac8b-188a20afe958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:35.766476 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d2b38-a118-4786-ac8b-188a20afe958 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:35.766828 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d2b38-a118-4786-ac8b-188a20afe958 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:35.767256 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d2b38-a118-4786-ac8b-188a20afe958 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:35.768438 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89d2b38-a118-4786-ac8b-188a20afe958 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:51:35.768438 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89d2b38-a118-4786-ac8b-188a20afe958 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:35.770207 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89d2b38-a118-4786-ac8b-188a20afe958 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:35.786784 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1006/2008] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jun 15 14:51:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:51:35.791570 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cca4002-d2b5-49ed-957c-968a97b64883 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:35.808551 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cca4002-d2b5-49ed-957c-968a97b64883 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:35.815897 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3724242-0793-45dc-8df6-63d074631854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:35.816664 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3724242-0793-45dc-8df6-63d074631854 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:35.816893 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3724242-0793-45dc-8df6-63d074631854 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:35.817334 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3724242-0793-45dc-8df6-63d074631854 admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:35.818813 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3724242-0793-45dc-8df6-63d074631854 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:35.821652 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3724242-0793-45dc-8df6-63d074631854 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:36.004341 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1003/2009] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:51:35 2022] POST /identity/v3/projects => generated 397 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:51:36.008429 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89d2b38-a118-4786-ac8b-188a20afe958 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:36.022520 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89d2b38-a118-4786-ac8b-188a20afe958 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:36.112837 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cc1682-9119-46ad-b2f6-d257f8c286eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:36.113644 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc1682-9119-46ad-b2f6-d257f8c286eb admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:36.113985 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc1682-9119-46ad-b2f6-d257f8c286eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:36.114328 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc1682-9119-46ad-b2f6-d257f8c286eb admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:36.115113 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cc1682-9119-46ad-b2f6-d257f8c286eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:36.117231 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cc1682-9119-46ad-b2f6-d257f8c286eb admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:36.201288 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3724242-0793-45dc-8df6-63d074631854 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:36.216595 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-a3724242-0793-45dc-8df6-63d074631854 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:36.216774 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1007/2010] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:51:36 2022] POST /identity/v3/users => generated 299 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:51:36.219667 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cc1682-9119-46ad-b2f6-d257f8c286eb admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:36.231841 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cc1682-9119-46ad-b2f6-d257f8c286eb admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:36.238187 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238536d8-01c4-45bf-94bd-9b2a1b353634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:36.238830 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238536d8-01c4-45bf-94bd-9b2a1b353634 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:36.239117 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238536d8-01c4-45bf-94bd-9b2a1b353634 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:36.239399 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238536d8-01c4-45bf-94bd-9b2a1b353634 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:36.240223 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238536d8-01c4-45bf-94bd-9b2a1b353634 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:36.243575 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238536d8-01c4-45bf-94bd-9b2a1b353634 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:36.256679 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1008/2011] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:51:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:51:36.261578 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238536d8-01c4-45bf-94bd-9b2a1b353634 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:36.271935 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238536d8-01c4-45bf-94bd-9b2a1b353634 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:36.279205 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cef0d5b-6097-45ac-9e86-40042ea7d54a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:36.280255 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef0d5b-6097-45ac-9e86-40042ea7d54a admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:36.280255 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef0d5b-6097-45ac-9e86-40042ea7d54a admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:36.280463 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef0d5b-6097-45ac-9e86-40042ea7d54a admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:36.281227 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cef0d5b-6097-45ac-9e86-40042ea7d54a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:36.282870 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cef0d5b-6097-45ac-9e86-40042ea7d54a admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:36.283624 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3724242-0793-45dc-8df6-63d074631854 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:36.291645 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11589f9d-46b6-4d98-9064-26be3a12ab0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:36.292416 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11589f9d-46b6-4d98-9064-26be3a12ab0c service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:36.292728 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11589f9d-46b6-4d98-9064-26be3a12ab0c service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:36.305025 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11589f9d-46b6-4d98-9064-26be3a12ab0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:36.305025 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-11589f9d-46b6-4d98-9064-26be3a12ab0c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:36.305250 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1009/2012] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:51:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:51:36.307862 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cef0d5b-6097-45ac-9e86-40042ea7d54a admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:36.319204 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cef0d5b-6097-45ac-9e86-40042ea7d54a admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:36.325349 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd428394-96cc-4481-a08f-441d3711d24d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:36.326310 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd428394-96cc-4481-a08f-441d3711d24d admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:36.327240 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd428394-96cc-4481-a08f-441d3711d24d admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:36.327240 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd428394-96cc-4481-a08f-441d3711d24d admin admin] PATH_INFO: `/v3/projects/a10c82cde1a3408f88d3fad161d58e66/users/6d7498aa2eed4c2da92785e5706c9cfc/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:36.337754 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:36.345870 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd428394-96cc-4481-a08f-441d3711d24d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a10c82cde1a3408f88d3fad161d58e66, user_id=6d7498aa2eed4c2da92785e5706c9cfc, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:36.348314 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd428394-96cc-4481-a08f-441d3711d24d admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:36.372284 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11589f9d-46b6-4d98-9064-26be3a12ab0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:36.373727 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11589f9d-46b6-4d98-9064-26be3a12ab0c service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:36.384119 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1004/2013] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:51:36 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:51:36.493035 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:36.510647 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1010/2014] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:51:36 2022] PUT /identity/v3/projects/a10c82cde1a3408f88d3fad161d58e66/users/6d7498aa2eed4c2da92785e5706c9cfc/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:51:36.520457 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae0275d-cdca-4dd0-ae65-6639011d1750 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:36.521382 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae0275d-cdca-4dd0-ae65-6639011d1750 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:36.521897 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae0275d-cdca-4dd0-ae65-6639011d1750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:36.565801 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-fae0275d-cdca-4dd0-ae65-6639011d1750 None None] MFA Rules not processed for user `6d7498aa2eed4c2da92785e5706c9cfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:36.567037 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-fae0275d-cdca-4dd0-ae65-6639011d1750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:36.637445 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1005/2015] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:51:36 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:36.887693 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:37.042494 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:37.123154 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:37.537186 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd428394-96cc-4481-a08f-441d3711d24d admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:37.543658 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-fd428394-96cc-4481-a08f-441d3711d24d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:37.635634 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd428394-96cc-4481-a08f-441d3711d24d admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:37.639655 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a70225d-7123-4be0-ad50-c850196ab5da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:37.640480 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a70225d-7123-4be0-ad50-c850196ab5da service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:37.641827 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a70225d-7123-4be0-ad50-c850196ab5da service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:37.641944 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a70225d-7123-4be0-ad50-c850196ab5da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:37.654058 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a70225d-7123-4be0-ad50-c850196ab5da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:37.655547 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a70225d-7123-4be0-ad50-c850196ab5da service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:37.669632 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1011/2016] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:51:37 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:51:37.695183 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:38.467888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:38.940300 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:39.336577 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:40.174485 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:40.375747 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae0275d-cdca-4dd0-ae65-6639011d1750 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:40.377446 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-fae0275d-cdca-4dd0-ae65-6639011d1750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:40.470391 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae0275d-cdca-4dd0-ae65-6639011d1750 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:40.475124 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b9d6dd-19e0-40ac-9132-001bc9acd712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:40.476772 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b9d6dd-19e0-40ac-9132-001bc9acd712 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:40.477139 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b9d6dd-19e0-40ac-9132-001bc9acd712 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:40.477411 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b9d6dd-19e0-40ac-9132-001bc9acd712 admin admin] PATH_INFO: `/v3/users/661bfb914c8046e3b035d17208ca2ec3` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:40.478847 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b9d6dd-19e0-40ac-9132-001bc9acd712 admin admin] RBAC: Authorizing `identity:delete_user(user_id=661bfb914c8046e3b035d17208ca2ec3)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:40.481757 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b9d6dd-19e0-40ac-9132-001bc9acd712 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:40.766989 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-14b9d6dd-19e0-40ac-9132-001bc9acd712 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '661bfb914c8046e3b035d17208ca2ec3'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:40.767691 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-14b9d6dd-19e0-40ac-9132-001bc9acd712 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '661bfb914c8046e3b035d17208ca2ec3'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:40.775793 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-14b9d6dd-19e0-40ac-9132-001bc9acd712 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '661bfb914c8046e3b035d17208ca2ec3'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:40.776545 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-14b9d6dd-19e0-40ac-9132-001bc9acd712 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '661bfb914c8046e3b035d17208ca2ec3'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:40.871809 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-14b9d6dd-19e0-40ac-9132-001bc9acd712 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '661bfb914c8046e3b035d17208ca2ec3'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:40.887739 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:40.891873 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1006/2017] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:51:40 2022] DELETE /identity/v3/users/661bfb914c8046e3b035d17208ca2ec3 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:40.896530 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dffcf2-a3c2-4510-820b-90cb86d2aaac None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:40.897023 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dffcf2-a3c2-4510-820b-90cb86d2aaac None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:40.897118 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dffcf2-a3c2-4510-820b-90cb86d2aaac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:40.918617 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-71dffcf2-a3c2-4510-820b-90cb86d2aaac None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:40.919990 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-71dffcf2-a3c2-4510-820b-90cb86d2aaac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:40.989945 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1012/2018] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:51:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:41.000530 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b9d6dd-19e0-40ac-9132-001bc9acd712 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:41.002307 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-14b9d6dd-19e0-40ac-9132-001bc9acd712 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:41.065130 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b9d6dd-19e0-40ac-9132-001bc9acd712 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:41.068938 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c16f70-b09f-4a6f-80b0-524a5aeefc69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:41.070165 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c16f70-b09f-4a6f-80b0-524a5aeefc69 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:41.070446 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c16f70-b09f-4a6f-80b0-524a5aeefc69 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:41.070720 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c16f70-b09f-4a6f-80b0-524a5aeefc69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:41.080199 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c16f70-b09f-4a6f-80b0-524a5aeefc69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:41.081639 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c16f70-b09f-4a6f-80b0-524a5aeefc69 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:41.092669 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1007/2019] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:51:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:51:41.591697 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dffcf2-a3c2-4510-820b-90cb86d2aaac None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:41.594039 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-71dffcf2-a3c2-4510-820b-90cb86d2aaac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:41.624737 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dffcf2-a3c2-4510-820b-90cb86d2aaac None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:41.630559 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dd6e14-e586-4182-a8d8-c573b015cd2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:41.631576 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dd6e14-e586-4182-a8d8-c573b015cd2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:41.632876 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dd6e14-e586-4182-a8d8-c573b015cd2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:41.633620 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dd6e14-e586-4182-a8d8-c573b015cd2f admin admin] PATH_INFO: `/v3/projects/3c7302bdd2074855a20fcbffa7811a40` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:41.635399 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dd6e14-e586-4182-a8d8-c573b015cd2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c7302bdd2074855a20fcbffa7811a40)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:41.637902 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dd6e14-e586-4182-a8d8-c573b015cd2f admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:41.765053 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c8dd6e14-e586-4182-a8d8-c573b015cd2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c7302bdd2074855a20fcbffa7811a40', 'request_id': 'req-c8dd6e14-e586-4182-a8d8-c573b015cd2f'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:41.772022 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c8dd6e14-e586-4182-a8d8-c573b015cd2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c7302bdd2074855a20fcbffa7811a40', 'request_id': 'req-c8dd6e14-e586-4182-a8d8-c573b015cd2f'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:41.871839 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c8dd6e14-e586-4182-a8d8-c573b015cd2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c7302bdd2074855a20fcbffa7811a40', 'request_id': 'req-c8dd6e14-e586-4182-a8d8-c573b015cd2f'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:41.885119 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c8dd6e14-e586-4182-a8d8-c573b015cd2f admin admin] The token cache is being invalidate because project 3c7302bdd2074855a20fcbffa7811a40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:51:41.885247 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c8dd6e14-e586-4182-a8d8-c573b015cd2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:41.886273 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c8dd6e14-e586-4182-a8d8-c573b015cd2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:41.887867 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1013/2020] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:51:41 2022] DELETE /identity/v3/projects/3c7302bdd2074855a20fcbffa7811a40 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:51:41.892294 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c16f70-b09f-4a6f-80b0-524a5aeefc69 service neutron] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:41.903018 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c16f70-b09f-4a6f-80b0-524a5aeefc69 service neutron] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:41.982744 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a272e1d-d92b-45e7-a702-fe00ddf4d381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:41.983774 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a272e1d-d92b-45e7-a702-fe00ddf4d381 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:41.984147 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a272e1d-d92b-45e7-a702-fe00ddf4d381 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:41.984426 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a272e1d-d92b-45e7-a702-fe00ddf4d381 admin admin] PATH_INFO: `/v3/users/0b2975db7cac44e1a55bfdb91fc3a278` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:41.990239 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a272e1d-d92b-45e7-a702-fe00ddf4d381 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b2975db7cac44e1a55bfdb91fc3a278)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:41.990239 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a272e1d-d92b-45e7-a702-fe00ddf4d381 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:42.340159 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-7a272e1d-d92b-45e7-a702-fe00ddf4d381 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b2975db7cac44e1a55bfdb91fc3a278'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:42.342476 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-7a272e1d-d92b-45e7-a702-fe00ddf4d381 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b2975db7cac44e1a55bfdb91fc3a278'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:42.351995 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-7a272e1d-d92b-45e7-a702-fe00ddf4d381 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b2975db7cac44e1a55bfdb91fc3a278'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:42.353117 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-7a272e1d-d92b-45e7-a702-fe00ddf4d381 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b2975db7cac44e1a55bfdb91fc3a278'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:42.478457 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-7a272e1d-d92b-45e7-a702-fe00ddf4d381 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b2975db7cac44e1a55bfdb91fc3a278'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:42.499970 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1008/2021] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:51:41 2022] DELETE /identity/v3/users/0b2975db7cac44e1a55bfdb91fc3a278 => generated 0 bytes in 608 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:43.087322 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dd6e14-e586-4182-a8d8-c573b015cd2f admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:43.095502 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c8dd6e14-e586-4182-a8d8-c573b015cd2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:43.113271 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dd6e14-e586-4182-a8d8-c573b015cd2f admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:43.180360 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:43.205273 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa95126a-4940-4ac6-be28-5820c1681dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:43.206527 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa95126a-4940-4ac6-be28-5820c1681dd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:43.207133 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa95126a-4940-4ac6-be28-5820c1681dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:43.207608 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa95126a-4940-4ac6-be28-5820c1681dd6 admin admin] PATH_INFO: `/v3/projects/886597d1bd9d45bf8f064e95687fd55c` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:43.211688 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa95126a-4940-4ac6-be28-5820c1681dd6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=886597d1bd9d45bf8f064e95687fd55c)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:43.213229 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa95126a-4940-4ac6-be28-5820c1681dd6 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:43.413259 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-aa95126a-4940-4ac6-be28-5820c1681dd6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '886597d1bd9d45bf8f064e95687fd55c', 'request_id': 'req-aa95126a-4940-4ac6-be28-5820c1681dd6'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:43.420685 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-aa95126a-4940-4ac6-be28-5820c1681dd6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '886597d1bd9d45bf8f064e95687fd55c', 'request_id': 'req-aa95126a-4940-4ac6-be28-5820c1681dd6'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:43.531692 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-aa95126a-4940-4ac6-be28-5820c1681dd6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '886597d1bd9d45bf8f064e95687fd55c', 'request_id': 'req-aa95126a-4940-4ac6-be28-5820c1681dd6'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:43.567745 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-aa95126a-4940-4ac6-be28-5820c1681dd6 admin admin] The token cache is being invalidate because project 886597d1bd9d45bf8f064e95687fd55c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:51:43.567745 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-aa95126a-4940-4ac6-be28-5820c1681dd6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:43.567745 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-aa95126a-4940-4ac6-be28-5820c1681dd6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:43.567745 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1014/2022] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:51:43 2022] DELETE /identity/v3/projects/886597d1bd9d45bf8f064e95687fd55c => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:43.571120 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b5834d-558a-4729-895f-fea90290691b None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:43.571120 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b5834d-558a-4729-895f-fea90290691b None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:43.571120 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b5834d-558a-4729-895f-fea90290691b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:43.595399 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-31b5834d-558a-4729-895f-fea90290691b None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:43.596709 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-31b5834d-558a-4729-895f-fea90290691b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:43.616920 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:43.671302 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1009/2023] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:51:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:51:43.680435 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b246ed-e6f4-432e-bbbc-d29ff6dde040 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:43.680737 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b246ed-e6f4-432e-bbbc-d29ff6dde040 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:43.681716 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b246ed-e6f4-432e-bbbc-d29ff6dde040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:43.715895 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-d2b246ed-e6f4-432e-bbbc-d29ff6dde040 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:43.717254 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-d2b246ed-e6f4-432e-bbbc-d29ff6dde040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:43.726502 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1015/2024] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:51:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:43.731667 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b5834d-558a-4729-895f-fea90290691b None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:43.746455 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b5834d-558a-4729-895f-fea90290691b None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:43.753446 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f356081-4173-4627-93dd-14072c4b6ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:43.754356 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f356081-4173-4627-93dd-14072c4b6ccc admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:43.754726 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f356081-4173-4627-93dd-14072c4b6ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:43.755031 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f356081-4173-4627-93dd-14072c4b6ccc admin admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:43.755858 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f356081-4173-4627-93dd-14072c4b6ccc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:51:43.756206 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f356081-4173-4627-93dd-14072c4b6ccc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:43.758543 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f356081-4173-4627-93dd-14072c4b6ccc admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:43.777039 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1010/2025] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jun 15 14:51:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:51:43.781677 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b246ed-e6f4-432e-bbbc-d29ff6dde040 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:43.793485 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b246ed-e6f4-432e-bbbc-d29ff6dde040 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:43.802698 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f5d813-ca73-44e4-912b-abc08bb13f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:43.803422 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f5d813-ca73-44e4-912b-abc08bb13f29 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:43.804428 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f5d813-ca73-44e4-912b-abc08bb13f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:43.804920 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f5d813-ca73-44e4-912b-abc08bb13f29 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:43.805799 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f5d813-ca73-44e4-912b-abc08bb13f29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:43.808812 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f5d813-ca73-44e4-912b-abc08bb13f29 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:43.893101 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1016/2026] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:51:43 2022] POST /identity/v3/projects => generated 371 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:51:43.898952 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f356081-4173-4627-93dd-14072c4b6ccc admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:43.909806 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f356081-4173-4627-93dd-14072c4b6ccc admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:44.001255 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5dcd42-b6f1-4708-beb5-c4281992b0d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:44.003224 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5dcd42-b6f1-4708-beb5-c4281992b0d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:44.003224 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5dcd42-b6f1-4708-beb5-c4281992b0d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:44.003224 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5dcd42-b6f1-4708-beb5-c4281992b0d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:44.005164 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5dcd42-b6f1-4708-beb5-c4281992b0d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:44.007755 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5dcd42-b6f1-4708-beb5-c4281992b0d5 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:44.118925 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1011/2027] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:51:43 2022] POST /identity/v3/users => generated 286 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:51:44.123958 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f5d813-ca73-44e4-912b-abc08bb13f29 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:44.139529 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f5d813-ca73-44e4-912b-abc08bb13f29 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:44.149480 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd73d778-8f9e-4db6-a361-9df8e6ed2a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:44.150337 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd73d778-8f9e-4db6-a361-9df8e6ed2a0c admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:44.150620 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd73d778-8f9e-4db6-a361-9df8e6ed2a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:44.150908 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd73d778-8f9e-4db6-a361-9df8e6ed2a0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:44.151640 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd73d778-8f9e-4db6-a361-9df8e6ed2a0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:44.153266 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd73d778-8f9e-4db6-a361-9df8e6ed2a0c admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:44.170687 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1017/2028] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:51:44 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:51:44.176056 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5dcd42-b6f1-4708-beb5-c4281992b0d5 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:44.191747 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5dcd42-b6f1-4708-beb5-c4281992b0d5 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:44.201810 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac09e4b8-ef84-4302-9190-43270e36648b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:44.201810 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac09e4b8-ef84-4302-9190-43270e36648b admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:44.201810 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac09e4b8-ef84-4302-9190-43270e36648b admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:44.201810 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac09e4b8-ef84-4302-9190-43270e36648b admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:44.202170 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac09e4b8-ef84-4302-9190-43270e36648b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:44.203344 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac09e4b8-ef84-4302-9190-43270e36648b admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:44.220190 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1012/2029] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:51:44 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:51:44.226108 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd73d778-8f9e-4db6-a361-9df8e6ed2a0c admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:44.239407 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd73d778-8f9e-4db6-a361-9df8e6ed2a0c admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:44.245922 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16bb6da-f230-45cd-9372-166cf21e145a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:44.245922 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16bb6da-f230-45cd-9372-166cf21e145a admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:44.245922 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16bb6da-f230-45cd-9372-166cf21e145a admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:44.246275 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16bb6da-f230-45cd-9372-166cf21e145a admin admin] PATH_INFO: `/v3/projects/5c81829dc7734625ba3180a963fed1a4/users/e642c5db4bcd4290ad9a1cedc0cdbabd/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:44.267481 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16bb6da-f230-45cd-9372-166cf21e145a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c81829dc7734625ba3180a963fed1a4, user_id=e642c5db4bcd4290ad9a1cedc0cdbabd, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:44.270244 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16bb6da-f230-45cd-9372-166cf21e145a admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:44.380926 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1018/2030] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:51:44 2022] PUT /identity/v3/projects/5c81829dc7734625ba3180a963fed1a4/users/e642c5db4bcd4290ad9a1cedc0cdbabd/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:44.392559 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe733b0-2920-4acd-9c3b-821b1d180a81 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:44.393019 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe733b0-2920-4acd-9c3b-821b1d180a81 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:44.393433 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe733b0-2920-4acd-9c3b-821b1d180a81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:44.439654 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-bbe733b0-2920-4acd-9c3b-821b1d180a81 None None] MFA Rules not processed for user `e642c5db4bcd4290ad9a1cedc0cdbabd`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:44.441175 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-bbe733b0-2920-4acd-9c3b-821b1d180a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:44.549551 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1013/2031] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:51:44 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:44.567351 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16bb6da-f230-45cd-9372-166cf21e145a admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:44.571712 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a16bb6da-f230-45cd-9372-166cf21e145a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:44.657161 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16bb6da-f230-45cd-9372-166cf21e145a admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:44.664378 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3443654-01aa-4fe1-a16a-fda422ccaea7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:44.665571 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3443654-01aa-4fe1-a16a-fda422ccaea7 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:44.666098 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3443654-01aa-4fe1-a16a-fda422ccaea7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:44.666616 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3443654-01aa-4fe1-a16a-fda422ccaea7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:44.678488 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3443654-01aa-4fe1-a16a-fda422ccaea7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:44.681095 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3443654-01aa-4fe1-a16a-fda422ccaea7 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:44.694753 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1019/2032] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:51:44 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:51:44.730578 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe733b0-2920-4acd-9c3b-821b1d180a81 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:44.752955 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:44.811479 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe733b0-2920-4acd-9c3b-821b1d180a81 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:44.816719 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7886b00e-5e26-4653-b02a-06b73b8032c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:44.817674 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7886b00e-5e26-4653-b02a-06b73b8032c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:44.817950 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7886b00e-5e26-4653-b02a-06b73b8032c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:44.818225 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7886b00e-5e26-4653-b02a-06b73b8032c7 admin admin] PATH_INFO: `/v3/users/e642c5db4bcd4290ad9a1cedc0cdbabd` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:44.820199 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7886b00e-5e26-4653-b02a-06b73b8032c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e642c5db4bcd4290ad9a1cedc0cdbabd)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:44.822451 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7886b00e-5e26-4653-b02a-06b73b8032c7 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:45.034497 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-7886b00e-5e26-4653-b02a-06b73b8032c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e642c5db4bcd4290ad9a1cedc0cdbabd'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:45.035170 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-7886b00e-5e26-4653-b02a-06b73b8032c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e642c5db4bcd4290ad9a1cedc0cdbabd'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:45.046285 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-7886b00e-5e26-4653-b02a-06b73b8032c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e642c5db4bcd4290ad9a1cedc0cdbabd'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:45.047096 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-7886b00e-5e26-4653-b02a-06b73b8032c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e642c5db4bcd4290ad9a1cedc0cdbabd'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:45.146474 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-7886b00e-5e26-4653-b02a-06b73b8032c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e642c5db4bcd4290ad9a1cedc0cdbabd'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:45.172790 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1014/2033] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:51:44 2022] DELETE /identity/v3/users/e642c5db4bcd4290ad9a1cedc0cdbabd => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:45.177553 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c4903-f2d2-46df-b1dd-14654e7496f0 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:45.178644 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c4903-f2d2-46df-b1dd-14654e7496f0 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:45.178644 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c4903-f2d2-46df-b1dd-14654e7496f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:45.201286 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-095c4903-f2d2-46df-b1dd-14654e7496f0 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:45.203492 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-095c4903-f2d2-46df-b1dd-14654e7496f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:45.219636 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:45.285208 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1020/2034] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:51:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:51:45.296617 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7886b00e-5e26-4653-b02a-06b73b8032c7 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:45.298294 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-7886b00e-5e26-4653-b02a-06b73b8032c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:45.370398 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7886b00e-5e26-4653-b02a-06b73b8032c7 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:45.374216 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab01477-ba08-4205-9027-c9d067d57b22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:45.375104 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab01477-ba08-4205-9027-c9d067d57b22 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:45.375252 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab01477-ba08-4205-9027-c9d067d57b22 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:45.375521 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab01477-ba08-4205-9027-c9d067d57b22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:45.385995 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab01477-ba08-4205-9027-c9d067d57b22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:45.387697 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab01477-ba08-4205-9027-c9d067d57b22 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:45.398980 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1015/2035] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:51:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:51:45.903020 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095c4903-f2d2-46df-b1dd-14654e7496f0 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:45.904851 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-095c4903-f2d2-46df-b1dd-14654e7496f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:45.924968 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095c4903-f2d2-46df-b1dd-14654e7496f0 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:45.932482 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0bf856-6850-4202-af7f-cfae47820a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:45.934815 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0bf856-6850-4202-af7f-cfae47820a96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:45.935477 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0bf856-6850-4202-af7f-cfae47820a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:45.936150 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0bf856-6850-4202-af7f-cfae47820a96 admin admin] PATH_INFO: `/v3/projects/5c81829dc7734625ba3180a963fed1a4` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:45.939086 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0bf856-6850-4202-af7f-cfae47820a96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c81829dc7734625ba3180a963fed1a4)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:45.943037 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0bf856-6850-4202-af7f-cfae47820a96 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:46.049072 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ef0bf856-6850-4202-af7f-cfae47820a96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c81829dc7734625ba3180a963fed1a4', 'request_id': 'req-ef0bf856-6850-4202-af7f-cfae47820a96'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:46.056997 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ef0bf856-6850-4202-af7f-cfae47820a96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c81829dc7734625ba3180a963fed1a4', 'request_id': 'req-ef0bf856-6850-4202-af7f-cfae47820a96'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:46.111782 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ef0bf856-6850-4202-af7f-cfae47820a96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c81829dc7734625ba3180a963fed1a4', 'request_id': 'req-ef0bf856-6850-4202-af7f-cfae47820a96'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:46.128299 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ef0bf856-6850-4202-af7f-cfae47820a96 admin admin] The token cache is being invalidate because project 5c81829dc7734625ba3180a963fed1a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:51:46.128590 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ef0bf856-6850-4202-af7f-cfae47820a96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:46.129256 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ef0bf856-6850-4202-af7f-cfae47820a96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:46.130866 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1021/2036] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:51:45 2022] DELETE /identity/v3/projects/5c81829dc7734625ba3180a963fed1a4 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:46.137193 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbdb19b-b707-4118-85f8-7443993db392 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:46.137942 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbdb19b-b707-4118-85f8-7443993db392 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:46.137942 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbdb19b-b707-4118-85f8-7443993db392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:46.159342 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-bcbdb19b-b707-4118-85f8-7443993db392 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:46.160751 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-bcbdb19b-b707-4118-85f8-7443993db392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:46.208949 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:46.251175 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1016/2037] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:51:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:46.261276 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c003021-7fbd-4b26-a58e-66b8a1caa5b6 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:46.261911 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c003021-7fbd-4b26-a58e-66b8a1caa5b6 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:46.262044 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c003021-7fbd-4b26-a58e-66b8a1caa5b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:46.283055 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-9c003021-7fbd-4b26-a58e-66b8a1caa5b6 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:46.288558 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-9c003021-7fbd-4b26-a58e-66b8a1caa5b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:46.293004 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1022/2038] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:51:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:46.298378 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbdb19b-b707-4118-85f8-7443993db392 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:46.309881 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbdb19b-b707-4118-85f8-7443993db392 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:46.315544 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bb5c80-1157-4d82-843f-f4dd95b31f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:46.316275 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bb5c80-1157-4d82-843f-f4dd95b31f1f admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:46.316760 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bb5c80-1157-4d82-843f-f4dd95b31f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:46.317327 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bb5c80-1157-4d82-843f-f4dd95b31f1f admin admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:46.317897 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bb5c80-1157-4d82-843f-f4dd95b31f1f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:51:46.318185 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bb5c80-1157-4d82-843f-f4dd95b31f1f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:46.321237 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bb5c80-1157-4d82-843f-f4dd95b31f1f admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:46.335384 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1017/2039] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jun 15 14:51:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:51:46.340164 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c003021-7fbd-4b26-a58e-66b8a1caa5b6 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:46.353821 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c003021-7fbd-4b26-a58e-66b8a1caa5b6 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:46.363824 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b7d50f-871c-44c0-b279-fa1b04141397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:46.364442 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b7d50f-871c-44c0-b279-fa1b04141397 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:46.364731 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b7d50f-871c-44c0-b279-fa1b04141397 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:46.365008 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b7d50f-871c-44c0-b279-fa1b04141397 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:46.365772 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b7d50f-871c-44c0-b279-fa1b04141397 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:46.369597 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b7d50f-871c-44c0-b279-fa1b04141397 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:46.499860 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1023/2040] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:51:46 2022] POST /identity/v3/projects => generated 391 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:51:46.503989 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bb5c80-1157-4d82-843f-f4dd95b31f1f admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:46.516267 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bb5c80-1157-4d82-843f-f4dd95b31f1f admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:46.599202 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f045967d-0fa8-445c-9ec9-af5f8974855b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:46.599883 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f045967d-0fa8-445c-9ec9-af5f8974855b admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:46.600206 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f045967d-0fa8-445c-9ec9-af5f8974855b admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:46.600523 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f045967d-0fa8-445c-9ec9-af5f8974855b admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:46.601236 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f045967d-0fa8-445c-9ec9-af5f8974855b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:46.603253 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f045967d-0fa8-445c-9ec9-af5f8974855b admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:46.765323 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1018/2041] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:51:46 2022] POST /identity/v3/users => generated 296 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:51:46.776911 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b7d50f-871c-44c0-b279-fa1b04141397 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:46.792359 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b7d50f-871c-44c0-b279-fa1b04141397 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:46.797557 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd58d9e-1f1f-461d-97a6-bbe4386d8502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:46.798255 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd58d9e-1f1f-461d-97a6-bbe4386d8502 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:46.798535 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd58d9e-1f1f-461d-97a6-bbe4386d8502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:46.798811 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd58d9e-1f1f-461d-97a6-bbe4386d8502 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:46.799554 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd58d9e-1f1f-461d-97a6-bbe4386d8502 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:46.801315 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd58d9e-1f1f-461d-97a6-bbe4386d8502 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:46.816234 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1024/2042] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:51:46 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:51:46.820869 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f045967d-0fa8-445c-9ec9-af5f8974855b admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:46.831168 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f045967d-0fa8-445c-9ec9-af5f8974855b admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:46.836547 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0ad92d-cde2-42f4-b9f6-959891181fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:46.837251 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ad92d-cde2-42f4-b9f6-959891181fb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:46.837536 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ad92d-cde2-42f4-b9f6-959891181fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:46.837810 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ad92d-cde2-42f4-b9f6-959891181fb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:46.838528 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0ad92d-cde2-42f4-b9f6-959891181fb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:46.840134 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0ad92d-cde2-42f4-b9f6-959891181fb8 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:46.850820 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1019/2043] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:51:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:51:46.855330 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd58d9e-1f1f-461d-97a6-bbe4386d8502 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:46.866555 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd58d9e-1f1f-461d-97a6-bbe4386d8502 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:46.871543 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ee0a27-6090-401f-80ce-828be12c4cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:46.872384 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ee0a27-6090-401f-80ce-828be12c4cb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:46.872671 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ee0a27-6090-401f-80ce-828be12c4cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:46.872944 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ee0a27-6090-401f-80ce-828be12c4cb4 admin admin] PATH_INFO: `/v3/projects/aa4bcabaca5641f3b5372df5332d948a/users/b165fd1ff9f94d42b139672a84f770b2/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:46.890684 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ee0a27-6090-401f-80ce-828be12c4cb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa4bcabaca5641f3b5372df5332d948a, user_id=b165fd1ff9f94d42b139672a84f770b2, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:46.893462 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ee0a27-6090-401f-80ce-828be12c4cb4 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:47.003596 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1025/2044] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:51:46 2022] PUT /identity/v3/projects/aa4bcabaca5641f3b5372df5332d948a/users/b165fd1ff9f94d42b139672a84f770b2/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:47.016408 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7fa5a0-1ecc-4169-ac0b-ee9e833abdc1 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:47.016725 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7fa5a0-1ecc-4169-ac0b-ee9e833abdc1 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:47.017009 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7fa5a0-1ecc-4169-ac0b-ee9e833abdc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:47.047304 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-ae7fa5a0-1ecc-4169-ac0b-ee9e833abdc1 None None] MFA Rules not processed for user `b165fd1ff9f94d42b139672a84f770b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:47.048577 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-ae7fa5a0-1ecc-4169-ac0b-ee9e833abdc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:47.134805 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1020/2045] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:51:47 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:47.146772 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ee0a27-6090-401f-80ce-828be12c4cb4 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:47.148660 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c1ee0a27-6090-401f-80ce-828be12c4cb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:47.193662 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:47.213328 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ee0a27-6090-401f-80ce-828be12c4cb4 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:47.217153 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5bf13f-8450-4803-937b-684f592daa1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:47.217838 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5bf13f-8450-4803-937b-684f592daa1b service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:47.218171 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5bf13f-8450-4803-937b-684f592daa1b service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:47.218448 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5bf13f-8450-4803-937b-684f592daa1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:47.227759 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5bf13f-8450-4803-937b-684f592daa1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:47.229355 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5bf13f-8450-4803-937b-684f592daa1b service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:47.240004 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1026/2046] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:51:47 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:51:54.705953 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7fa5a0-1ecc-4169-ac0b-ee9e833abdc1 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:54.707400 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-ae7fa5a0-1ecc-4169-ac0b-ee9e833abdc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:54.767468 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:54.888610 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7fa5a0-1ecc-4169-ac0b-ee9e833abdc1 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:54.894166 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd6b343-265f-4a53-8c6d-da6c2d6ef39c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:54.897391 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd6b343-265f-4a53-8c6d-da6c2d6ef39c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:54.897859 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd6b343-265f-4a53-8c6d-da6c2d6ef39c admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:54.898903 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd6b343-265f-4a53-8c6d-da6c2d6ef39c admin admin] PATH_INFO: `/v3/users/9ce606720c304964b46b6786ee829edb` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:54.900679 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd6b343-265f-4a53-8c6d-da6c2d6ef39c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ce606720c304964b46b6786ee829edb)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:54.902621 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd6b343-265f-4a53-8c6d-da6c2d6ef39c admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:55.240037 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-8dd6b343-265f-4a53-8c6d-da6c2d6ef39c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ce606720c304964b46b6786ee829edb'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:55.240856 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-8dd6b343-265f-4a53-8c6d-da6c2d6ef39c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ce606720c304964b46b6786ee829edb'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:55.253763 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-8dd6b343-265f-4a53-8c6d-da6c2d6ef39c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ce606720c304964b46b6786ee829edb'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:55.254663 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-8dd6b343-265f-4a53-8c6d-da6c2d6ef39c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ce606720c304964b46b6786ee829edb'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:55.273121 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5bf13f-8450-4803-937b-684f592daa1b service cinder] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:55.277354 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-ac5bf13f-8450-4803-937b-684f592daa1b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:55.324980 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5bf13f-8450-4803-937b-684f592daa1b service cinder] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:55.332424 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a57db2a-7035-4538-8a35-5c012a33d0aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:55.333817 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a57db2a-7035-4538-8a35-5c012a33d0aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:55.334273 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a57db2a-7035-4538-8a35-5c012a33d0aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:55.334754 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a57db2a-7035-4538-8a35-5c012a33d0aa admin admin] PATH_INFO: `/v3/users/76cee99357614bdeb4b4377161dcc674` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:55.336921 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a57db2a-7035-4538-8a35-5c012a33d0aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=76cee99357614bdeb4b4377161dcc674)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:55.340230 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a57db2a-7035-4538-8a35-5c012a33d0aa admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:55.381258 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-8dd6b343-265f-4a53-8c6d-da6c2d6ef39c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ce606720c304964b46b6786ee829edb'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:55.406420 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1021/2047] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:51:54 2022] DELETE /identity/v3/users/9ce606720c304964b46b6786ee829edb => generated 0 bytes in 701 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:55.407360 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd6b343-265f-4a53-8c6d-da6c2d6ef39c admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:55.408771 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-8dd6b343-265f-4a53-8c6d-da6c2d6ef39c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:55.497921 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd6b343-265f-4a53-8c6d-da6c2d6ef39c admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:55.505176 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da34126-a482-4146-845c-d3bb843d0abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:55.506235 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da34126-a482-4146-845c-d3bb843d0abc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:55.506527 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da34126-a482-4146-845c-d3bb843d0abc admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:55.506806 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da34126-a482-4146-845c-d3bb843d0abc admin admin] PATH_INFO: `/v3/users/6d7498aa2eed4c2da92785e5706c9cfc` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:55.509097 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da34126-a482-4146-845c-d3bb843d0abc admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d7498aa2eed4c2da92785e5706c9cfc)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:55.511698 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da34126-a482-4146-845c-d3bb843d0abc admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:55.530228 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8a57db2a-7035-4538-8a35-5c012a33d0aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76cee99357614bdeb4b4377161dcc674'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:55.530896 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8a57db2a-7035-4538-8a35-5c012a33d0aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76cee99357614bdeb4b4377161dcc674'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:55.531575 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:55.531705 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8a57db2a-7035-4538-8a35-5c012a33d0aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76cee99357614bdeb4b4377161dcc674'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:55.601322 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8a57db2a-7035-4538-8a35-5c012a33d0aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76cee99357614bdeb4b4377161dcc674'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:55.615467 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8a57db2a-7035-4538-8a35-5c012a33d0aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76cee99357614bdeb4b4377161dcc674'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:55.629682 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1027/2048] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:51:55 2022] DELETE /identity/v3/users/76cee99357614bdeb4b4377161dcc674 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:51:55.631713 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef18ccc-48e9-47dd-977d-dae59884749b None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:55.632236 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef18ccc-48e9-47dd-977d-dae59884749b None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:55.633224 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef18ccc-48e9-47dd-977d-dae59884749b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:55.653653 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-9ef18ccc-48e9-47dd-977d-dae59884749b None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:55.654858 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-9ef18ccc-48e9-47dd-977d-dae59884749b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:55.719706 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1028/2049] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:51:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:55.721130 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7faaad-c798-49b6-88ae-7cedcc66f79a None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:55.721421 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7faaad-c798-49b6-88ae-7cedcc66f79a None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:55.721705 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7faaad-c798-49b6-88ae-7cedcc66f79a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:55.740856 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-9d7faaad-c798-49b6-88ae-7cedcc66f79a None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:55.742591 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-9d7faaad-c798-49b6-88ae-7cedcc66f79a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:55.750671 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1029/2050] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:51:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:55.751492 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7faaad-c798-49b6-88ae-7cedcc66f79a None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:55.752796 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-9d7faaad-c798-49b6-88ae-7cedcc66f79a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:55.788895 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:55.835421 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-2da34126-a482-4146-845c-d3bb843d0abc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d7498aa2eed4c2da92785e5706c9cfc'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:55.836729 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-2da34126-a482-4146-845c-d3bb843d0abc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d7498aa2eed4c2da92785e5706c9cfc'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:55.843848 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-2da34126-a482-4146-845c-d3bb843d0abc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d7498aa2eed4c2da92785e5706c9cfc'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:55.844600 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-2da34126-a482-4146-845c-d3bb843d0abc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d7498aa2eed4c2da92785e5706c9cfc'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:55.846674 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7faaad-c798-49b6-88ae-7cedcc66f79a None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:55.850333 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2225cb06-f2aa-43d0-b184-b605fbbfde11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:55.850981 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2225cb06-f2aa-43d0-b184-b605fbbfde11 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:55.851261 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2225cb06-f2aa-43d0-b184-b605fbbfde11 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:55.851678 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2225cb06-f2aa-43d0-b184-b605fbbfde11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:55.853583 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-2225cb06-f2aa-43d0-b184-b605fbbfde11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:55.866265 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2225cb06-f2aa-43d0-b184-b605fbbfde11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:55.869353 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2225cb06-f2aa-43d0-b184-b605fbbfde11 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:55.879257 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1030/2051] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:51:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:51:55.880081 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2225cb06-f2aa-43d0-b184-b605fbbfde11 service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:55.881428 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-2225cb06-f2aa-43d0-b184-b605fbbfde11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:55.895420 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2225cb06-f2aa-43d0-b184-b605fbbfde11 service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:55.901296 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8d9e23-bca2-4849-8952-c08d1f0db6f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:55.901566 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8d9e23-bca2-4849-8952-c08d1f0db6f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:55.901806 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8d9e23-bca2-4849-8952-c08d1f0db6f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:55.902030 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8d9e23-bca2-4849-8952-c08d1f0db6f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:55.904168 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-ff8d9e23-bca2-4849-8952-c08d1f0db6f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:55.922579 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8d9e23-bca2-4849-8952-c08d1f0db6f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:55.924879 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8d9e23-bca2-4849-8952-c08d1f0db6f6 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:55.940273 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1031/2052] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:51:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:51:56.003344 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-2da34126-a482-4146-845c-d3bb843d0abc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d7498aa2eed4c2da92785e5706c9cfc'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:56.041040 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1022/2053] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:51:55 2022] DELETE /identity/v3/users/6d7498aa2eed4c2da92785e5706c9cfc => generated 0 bytes in 634 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:56.045838 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c287d457-972e-44f3-aff5-efe24832765b None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:56.049389 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c287d457-972e-44f3-aff5-efe24832765b None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:56.049389 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c287d457-972e-44f3-aff5-efe24832765b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:56.068794 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-c287d457-972e-44f3-aff5-efe24832765b None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:56.072927 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c287d457-972e-44f3-aff5-efe24832765b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:56.181081 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1032/2054] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:51:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:56.198853 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da34126-a482-4146-845c-d3bb843d0abc admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:56.268619 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da34126-a482-4146-845c-d3bb843d0abc admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:56.272769 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7c5fe6-31aa-4370-b401-d8a310827eb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:56.274183 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7c5fe6-31aa-4370-b401-d8a310827eb2 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:56.274797 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7c5fe6-31aa-4370-b401-d8a310827eb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:56.275198 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7c5fe6-31aa-4370-b401-d8a310827eb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:56.289078 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7c5fe6-31aa-4370-b401-d8a310827eb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:56.291988 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7c5fe6-31aa-4370-b401-d8a310827eb2 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:56.315856 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1023/2055] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:51:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:51:56.589834 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c287d457-972e-44f3-aff5-efe24832765b None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:56.591400 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c287d457-972e-44f3-aff5-efe24832765b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:56.611400 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c287d457-972e-44f3-aff5-efe24832765b None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:56.615806 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7c5fe6-31aa-4370-b401-d8a310827eb2 service neutron] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:56.617692 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-0c7c5fe6-31aa-4370-b401-d8a310827eb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:56.631282 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad41956-40b9-4cd2-81a5-6dda28bba22b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:56.632256 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad41956-40b9-4cd2-81a5-6dda28bba22b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:56.632540 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad41956-40b9-4cd2-81a5-6dda28bba22b admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:56.633432 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad41956-40b9-4cd2-81a5-6dda28bba22b admin admin] PATH_INFO: `/v3/projects/69ba2b1537c24d51b726164a1f6b0a05` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:56.635575 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad41956-40b9-4cd2-81a5-6dda28bba22b admin admin] RBAC: Authorizing `identity:delete_project(project_id=69ba2b1537c24d51b726164a1f6b0a05)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:56.636392 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7c5fe6-31aa-4370-b401-d8a310827eb2 service neutron] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:56.638307 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad41956-40b9-4cd2-81a5-6dda28bba22b admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:56.641887 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4335a5a-e1de-4d92-9cf8-df0d4d15ba44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:56.642770 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4335a5a-e1de-4d92-9cf8-df0d4d15ba44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:56.643052 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4335a5a-e1de-4d92-9cf8-df0d4d15ba44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:56.643325 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4335a5a-e1de-4d92-9cf8-df0d4d15ba44 admin admin] PATH_INFO: `/v3/projects/d72406625c194437852460ba8f03918e` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:56.644640 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4335a5a-e1de-4d92-9cf8-df0d4d15ba44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d72406625c194437852460ba8f03918e)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:56.646704 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4335a5a-e1de-4d92-9cf8-df0d4d15ba44 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:56.781222 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8ad41956-40b9-4cd2-81a5-6dda28bba22b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69ba2b1537c24d51b726164a1f6b0a05', 'request_id': 'req-8ad41956-40b9-4cd2-81a5-6dda28bba22b'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:56.787826 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8ad41956-40b9-4cd2-81a5-6dda28bba22b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69ba2b1537c24d51b726164a1f6b0a05', 'request_id': 'req-8ad41956-40b9-4cd2-81a5-6dda28bba22b'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:56.811963 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-f4335a5a-e1de-4d92-9cf8-df0d4d15ba44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd72406625c194437852460ba8f03918e', 'request_id': 'req-f4335a5a-e1de-4d92-9cf8-df0d4d15ba44'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:56.918113 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8ad41956-40b9-4cd2-81a5-6dda28bba22b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69ba2b1537c24d51b726164a1f6b0a05', 'request_id': 'req-8ad41956-40b9-4cd2-81a5-6dda28bba22b'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:56.931160 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8ad41956-40b9-4cd2-81a5-6dda28bba22b admin admin] The token cache is being invalidate because project 69ba2b1537c24d51b726164a1f6b0a05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:51:56.931461 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8ad41956-40b9-4cd2-81a5-6dda28bba22b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:56.932178 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8ad41956-40b9-4cd2-81a5-6dda28bba22b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:56.933863 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1033/2056] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:51:56 2022] DELETE /identity/v3/projects/69ba2b1537c24d51b726164a1f6b0a05 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:51:56.938700 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56722597-aad1-4188-823c-02abfa17e736 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:56.938990 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56722597-aad1-4188-823c-02abfa17e736 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:56.939269 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56722597-aad1-4188-823c-02abfa17e736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:56.959764 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-56722597-aad1-4188-823c-02abfa17e736 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:56.961810 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-56722597-aad1-4188-823c-02abfa17e736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:56.971782 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:57.028695 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1034/2057] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:51:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:51:57.029601 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56722597-aad1-4188-823c-02abfa17e736 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:57.030889 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-56722597-aad1-4188-823c-02abfa17e736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:57.041992 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-f4335a5a-e1de-4d92-9cf8-df0d4d15ba44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd72406625c194437852460ba8f03918e', 'request_id': 'req-f4335a5a-e1de-4d92-9cf8-df0d4d15ba44'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:57.043884 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-f4335a5a-e1de-4d92-9cf8-df0d4d15ba44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd72406625c194437852460ba8f03918e', 'request_id': 'req-f4335a5a-e1de-4d92-9cf8-df0d4d15ba44'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:57.045108 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56722597-aad1-4188-823c-02abfa17e736 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:57.050571 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca5ea29-14ad-45b2-a5e4-8abd2c488112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:57.051902 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca5ea29-14ad-45b2-a5e4-8abd2c488112 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:57.052106 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca5ea29-14ad-45b2-a5e4-8abd2c488112 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:57.052482 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca5ea29-14ad-45b2-a5e4-8abd2c488112 admin admin] PATH_INFO: `/v3/projects/a10c82cde1a3408f88d3fad161d58e66` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:57.053925 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca5ea29-14ad-45b2-a5e4-8abd2c488112 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a10c82cde1a3408f88d3fad161d58e66)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:57.056159 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca5ea29-14ad-45b2-a5e4-8abd2c488112 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:57.057161 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-f4335a5a-e1de-4d92-9cf8-df0d4d15ba44 admin admin] The token cache is being invalidate because project d72406625c194437852460ba8f03918e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:51:57.057456 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-f4335a5a-e1de-4d92-9cf8-df0d4d15ba44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:57.058120 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-f4335a5a-e1de-4d92-9cf8-df0d4d15ba44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:57.059706 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1024/2058] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:51:56 2022] DELETE /identity/v3/projects/d72406625c194437852460ba8f03918e => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:51:57.061053 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34f4081-7f4f-4f7d-8cf5-65c51f23300f None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:57.061343 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34f4081-7f4f-4f7d-8cf5-65c51f23300f None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:57.062197 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34f4081-7f4f-4f7d-8cf5-65c51f23300f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:57.080947 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-a34f4081-7f4f-4f7d-8cf5-65c51f23300f None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:57.082184 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-a34f4081-7f4f-4f7d-8cf5-65c51f23300f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:57.093882 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1025/2059] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:51:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:57.094724 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34f4081-7f4f-4f7d-8cf5-65c51f23300f None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:57.098759 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-a34f4081-7f4f-4f7d-8cf5-65c51f23300f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:57.108275 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34f4081-7f4f-4f7d-8cf5-65c51f23300f None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:57.118726 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15afacd9-27d4-4031-a1f2-8fb57a9dfa4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:57.119613 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15afacd9-27d4-4031-a1f2-8fb57a9dfa4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:57.119902 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15afacd9-27d4-4031-a1f2-8fb57a9dfa4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:57.120221 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15afacd9-27d4-4031-a1f2-8fb57a9dfa4a admin admin] PATH_INFO: `/v3/users/272317017d1e42f5b681dbdd6fe75538` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:57.121420 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15afacd9-27d4-4031-a1f2-8fb57a9dfa4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=272317017d1e42f5b681dbdd6fe75538)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:57.123488 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15afacd9-27d4-4031-a1f2-8fb57a9dfa4a admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:57.126058 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:57.158650 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-7ca5ea29-14ad-45b2-a5e4-8abd2c488112 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a10c82cde1a3408f88d3fad161d58e66', 'request_id': 'req-7ca5ea29-14ad-45b2-a5e4-8abd2c488112'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:57.166038 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-7ca5ea29-14ad-45b2-a5e4-8abd2c488112 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a10c82cde1a3408f88d3fad161d58e66', 'request_id': 'req-7ca5ea29-14ad-45b2-a5e4-8abd2c488112'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:57.281380 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-7ca5ea29-14ad-45b2-a5e4-8abd2c488112 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a10c82cde1a3408f88d3fad161d58e66', 'request_id': 'req-7ca5ea29-14ad-45b2-a5e4-8abd2c488112'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:57.292915 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-7ca5ea29-14ad-45b2-a5e4-8abd2c488112 admin admin] The token cache is being invalidate because project a10c82cde1a3408f88d3fad161d58e66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:51:57.293077 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-7ca5ea29-14ad-45b2-a5e4-8abd2c488112 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:57.293736 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-7ca5ea29-14ad-45b2-a5e4-8abd2c488112 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:57.295360 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1035/2060] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:51:57 2022] DELETE /identity/v3/projects/a10c82cde1a3408f88d3fad161d58e66 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:57.296145 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca5ea29-14ad-45b2-a5e4-8abd2c488112 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:57.297622 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-7ca5ea29-14ad-45b2-a5e4-8abd2c488112 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:57.329578 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-15afacd9-27d4-4031-a1f2-8fb57a9dfa4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '272317017d1e42f5b681dbdd6fe75538'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:57.330569 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-15afacd9-27d4-4031-a1f2-8fb57a9dfa4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '272317017d1e42f5b681dbdd6fe75538'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:57.340964 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-15afacd9-27d4-4031-a1f2-8fb57a9dfa4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '272317017d1e42f5b681dbdd6fe75538'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:57.343069 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-15afacd9-27d4-4031-a1f2-8fb57a9dfa4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '272317017d1e42f5b681dbdd6fe75538'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:57.377213 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca5ea29-14ad-45b2-a5e4-8abd2c488112 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:57.382465 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559acfd1-46b2-45b9-acd2-7aa73ca90b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:57.383962 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559acfd1-46b2-45b9-acd2-7aa73ca90b16 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:57.384285 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559acfd1-46b2-45b9-acd2-7aa73ca90b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:57.384566 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559acfd1-46b2-45b9-acd2-7aa73ca90b16 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:57.385335 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559acfd1-46b2-45b9-acd2-7aa73ca90b16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:51:57.385645 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559acfd1-46b2-45b9-acd2-7aa73ca90b16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:57.387552 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559acfd1-46b2-45b9-acd2-7aa73ca90b16 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:57.400712 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1036/2061] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jun 15 14:51:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:51:57.405958 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559acfd1-46b2-45b9-acd2-7aa73ca90b16 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:57.407797 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-559acfd1-46b2-45b9-acd2-7aa73ca90b16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:57.421584 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559acfd1-46b2-45b9-acd2-7aa73ca90b16 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:57.427158 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2c3163-60a4-4866-8b36-c9f58259f98d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:57.427832 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2c3163-60a4-4866-8b36-c9f58259f98d admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:57.428143 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2c3163-60a4-4866-8b36-c9f58259f98d admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:57.428567 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2c3163-60a4-4866-8b36-c9f58259f98d admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:57.429290 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2c3163-60a4-4866-8b36-c9f58259f98d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:57.432244 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2c3163-60a4-4866-8b36-c9f58259f98d admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:57.439696 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-15afacd9-27d4-4031-a1f2-8fb57a9dfa4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '272317017d1e42f5b681dbdd6fe75538'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:57.470815 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1026/2062] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:51:57 2022] DELETE /identity/v3/users/272317017d1e42f5b681dbdd6fe75538 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:57.556118 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1037/2063] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:51:57 2022] POST /identity/v3/projects => generated 369 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:51:57.558978 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15afacd9-27d4-4031-a1f2-8fb57a9dfa4a admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:57.688253 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15afacd9-27d4-4031-a1f2-8fb57a9dfa4a admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:57.752414 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b033960-77f1-4f2c-a114-5b06aba5a5a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:57.753106 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b033960-77f1-4f2c-a114-5b06aba5a5a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:57.753392 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b033960-77f1-4f2c-a114-5b06aba5a5a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:57.753793 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b033960-77f1-4f2c-a114-5b06aba5a5a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:57.754415 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b033960-77f1-4f2c-a114-5b06aba5a5a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:57.765035 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b033960-77f1-4f2c-a114-5b06aba5a5a8 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:57.853650 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1027/2064] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:51:57 2022] POST /identity/v3/users => generated 285 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:51:57.855266 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2c3163-60a4-4866-8b36-c9f58259f98d admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:57.868269 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2c3163-60a4-4866-8b36-c9f58259f98d admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:57.873385 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8031f42-b131-4bb6-b068-da4397d2c9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:57.874070 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8031f42-b131-4bb6-b068-da4397d2c9ae admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:57.874354 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8031f42-b131-4bb6-b068-da4397d2c9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:57.875103 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8031f42-b131-4bb6-b068-da4397d2c9ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:57.875364 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8031f42-b131-4bb6-b068-da4397d2c9ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:57.877147 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8031f42-b131-4bb6-b068-da4397d2c9ae admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:57.879732 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:57.893366 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1038/2065] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:51:57 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:51:57.895728 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b033960-77f1-4f2c-a114-5b06aba5a5a8 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:57.906130 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b033960-77f1-4f2c-a114-5b06aba5a5a8 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:57.911061 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7450acc-5ede-4d6a-abe3-ca956b6a4554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:57.911777 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7450acc-5ede-4d6a-abe3-ca956b6a4554 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:57.912061 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7450acc-5ede-4d6a-abe3-ca956b6a4554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:57.912547 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7450acc-5ede-4d6a-abe3-ca956b6a4554 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:57.913289 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7450acc-5ede-4d6a-abe3-ca956b6a4554 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:57.915471 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7450acc-5ede-4d6a-abe3-ca956b6a4554 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:57.926231 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1028/2066] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:51:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:51:57.931285 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8031f42-b131-4bb6-b068-da4397d2c9ae admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:57.941795 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8031f42-b131-4bb6-b068-da4397d2c9ae admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:57.947274 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf8f2e9-363a-4c58-9aee-c1a4c10630c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:57.948252 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf8f2e9-363a-4c58-9aee-c1a4c10630c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:57.948535 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf8f2e9-363a-4c58-9aee-c1a4c10630c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:57.948812 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf8f2e9-363a-4c58-9aee-c1a4c10630c4 admin admin] PATH_INFO: `/v3/projects/e8581254187e4316a0f0e1a2f2cd1bf1/users/74a47ed46ec1496fb6b9b27748c300d3/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:57.964623 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf8f2e9-363a-4c58-9aee-c1a4c10630c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8581254187e4316a0f0e1a2f2cd1bf1, user_id=74a47ed46ec1496fb6b9b27748c300d3, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:57.967292 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf8f2e9-363a-4c58-9aee-c1a4c10630c4 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:58.023641 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7450acc-5ede-4d6a-abe3-ca956b6a4554 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:58.023641 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-f7450acc-5ede-4d6a-abe3-ca956b6a4554 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:58.039509 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7450acc-5ede-4d6a-abe3-ca956b6a4554 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:58.045658 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7903c8-ee97-43e5-a4dc-222ecaa4d05f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:58.046543 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7903c8-ee97-43e5-a4dc-222ecaa4d05f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:58.046823 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7903c8-ee97-43e5-a4dc-222ecaa4d05f admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:58.047526 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7903c8-ee97-43e5-a4dc-222ecaa4d05f admin admin] PATH_INFO: `/v3/projects/2137841a71074dd482a694eaeb4803e3` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:58.052491 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7903c8-ee97-43e5-a4dc-222ecaa4d05f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2137841a71074dd482a694eaeb4803e3)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:58.052864 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1039/2067] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:51:57 2022] PUT /identity/v3/projects/e8581254187e4316a0f0e1a2f2cd1bf1/users/74a47ed46ec1496fb6b9b27748c300d3/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:58.052945 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7903c8-ee97-43e5-a4dc-222ecaa4d05f admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:58.052945 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:58.063059 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb999678-f2e0-409e-84c5-b43afc6153a3 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:58.063253 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb999678-f2e0-409e-84c5-b43afc6153a3 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:58.063526 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb999678-f2e0-409e-84c5-b43afc6153a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:58.095004 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-fb999678-f2e0-409e-84c5-b43afc6153a3 None None] MFA Rules not processed for user `74a47ed46ec1496fb6b9b27748c300d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:58.096289 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-fb999678-f2e0-409e-84c5-b43afc6153a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:58.159588 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-2f7903c8-ee97-43e5-a4dc-222ecaa4d05f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2137841a71074dd482a694eaeb4803e3', 'request_id': 'req-2f7903c8-ee97-43e5-a4dc-222ecaa4d05f'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:58.167366 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1040/2068] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:51:58 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:51:58.180015 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb999678-f2e0-409e-84c5-b43afc6153a3 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:58.181649 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-fb999678-f2e0-409e-84c5-b43afc6153a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:58.241477 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb999678-f2e0-409e-84c5-b43afc6153a3 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:58.248535 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-2f7903c8-ee97-43e5-a4dc-222ecaa4d05f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2137841a71074dd482a694eaeb4803e3', 'request_id': 'req-2f7903c8-ee97-43e5-a4dc-222ecaa4d05f'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:58.254520 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-2f7903c8-ee97-43e5-a4dc-222ecaa4d05f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2137841a71074dd482a694eaeb4803e3', 'request_id': 'req-2f7903c8-ee97-43e5-a4dc-222ecaa4d05f'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:58.267579 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-2f7903c8-ee97-43e5-a4dc-222ecaa4d05f admin admin] The token cache is being invalidate because project 2137841a71074dd482a694eaeb4803e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:51:58.267871 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-2f7903c8-ee97-43e5-a4dc-222ecaa4d05f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:58.268581 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-2f7903c8-ee97-43e5-a4dc-222ecaa4d05f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:58.270213 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1029/2069] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:51:58 2022] DELETE /identity/v3/projects/2137841a71074dd482a694eaeb4803e3 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:51:58.275532 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0555c12-fe96-4ec4-9233-4f844adddf9b None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:58.276059 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0555c12-fe96-4ec4-9233-4f844adddf9b None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:58.276521 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0555c12-fe96-4ec4-9233-4f844adddf9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:58.293560 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86268345-ca34-4467-9948-8f00641ccaef service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f18c57f27784694935bfb0bcea48a78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:58.294234 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86268345-ca34-4467-9948-8f00641ccaef service glance] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:58.294515 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86268345-ca34-4467-9948-8f00641ccaef service glance] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:58.294788 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86268345-ca34-4467-9948-8f00641ccaef service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:58.296543 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-f0555c12-fe96-4ec4-9233-4f844adddf9b None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:58.296655 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-86268345-ca34-4467-9948-8f00641ccaef service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:58.297727 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-f0555c12-fe96-4ec4-9233-4f844adddf9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:58.335302 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:58.368659 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86268345-ca34-4467-9948-8f00641ccaef service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:58.369803 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1030/2070] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:51:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:51:58.370709 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86268345-ca34-4467-9948-8f00641ccaef service glance] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:58.378363 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b184c-c15d-4f24-932a-43163abfe1ae None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:58.378825 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b184c-c15d-4f24-932a-43163abfe1ae None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:58.379288 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b184c-c15d-4f24-932a-43163abfe1ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:58.386930 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1041/2071] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:51:58 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:51:58.402707 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-7b4b184c-c15d-4f24-932a-43163abfe1ae None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:58.403952 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-7b4b184c-c15d-4f24-932a-43163abfe1ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:58.412063 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1031/2072] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:51:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:58.417440 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86268345-ca34-4467-9948-8f00641ccaef service glance] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:58.428224 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86268345-ca34-4467-9948-8f00641ccaef service glance] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:58.436888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79fe4e1-fb25-4000-8b38-2227468d01f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:58.436888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79fe4e1-fb25-4000-8b38-2227468d01f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:58.436888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79fe4e1-fb25-4000-8b38-2227468d01f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:58.436888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79fe4e1-fb25-4000-8b38-2227468d01f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:58.437237 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79fe4e1-fb25-4000-8b38-2227468d01f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:51:58.437237 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79fe4e1-fb25-4000-8b38-2227468d01f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:58.437917 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79fe4e1-fb25-4000-8b38-2227468d01f2 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:58.453922 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1042/2073] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jun 15 14:51:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:51:58.454993 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b184c-c15d-4f24-932a-43163abfe1ae None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:58.464740 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b184c-c15d-4f24-932a-43163abfe1ae None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:58.469342 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb315a3-1f1a-4178-aecb-413d7a386e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:58.471113 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb315a3-1f1a-4178-aecb-413d7a386e16 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:58.471113 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb315a3-1f1a-4178-aecb-413d7a386e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:58.471113 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb315a3-1f1a-4178-aecb-413d7a386e16 admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:58.471730 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb315a3-1f1a-4178-aecb-413d7a386e16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:58.473139 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb315a3-1f1a-4178-aecb-413d7a386e16 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:58.535975 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1032/2074] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:51:58 2022] POST /identity/v3/projects => generated 383 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:51:58.541199 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79fe4e1-fb25-4000-8b38-2227468d01f2 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:58.552238 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79fe4e1-fb25-4000-8b38-2227468d01f2 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:58.616993 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0380b83-ed35-4ba5-891c-d6998c16331e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:58.618764 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0380b83-ed35-4ba5-891c-d6998c16331e admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:58.618764 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0380b83-ed35-4ba5-891c-d6998c16331e admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:58.618960 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0380b83-ed35-4ba5-891c-d6998c16331e admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:58.619490 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0380b83-ed35-4ba5-891c-d6998c16331e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:58.621565 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0380b83-ed35-4ba5-891c-d6998c16331e admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:58.725713 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1043/2075] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:51:58 2022] POST /identity/v3/users => generated 292 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:51:58.727554 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb315a3-1f1a-4178-aecb-413d7a386e16 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:58.738902 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb315a3-1f1a-4178-aecb-413d7a386e16 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:58.745084 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c99e54f-218e-48df-b9f3-a8a21be9df5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:58.745764 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c99e54f-218e-48df-b9f3-a8a21be9df5d admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:58.746042 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c99e54f-218e-48df-b9f3-a8a21be9df5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:58.746313 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c99e54f-218e-48df-b9f3-a8a21be9df5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:58.747089 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c99e54f-218e-48df-b9f3-a8a21be9df5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:58.748728 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c99e54f-218e-48df-b9f3-a8a21be9df5d admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:58.760833 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1033/2076] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:51:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:51:58.765302 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0380b83-ed35-4ba5-891c-d6998c16331e admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:58.777597 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0380b83-ed35-4ba5-891c-d6998c16331e admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:58.783313 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4812437-86cb-45ac-be79-611bbb8f8ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:58.784118 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4812437-86cb-45ac-be79-611bbb8f8ec3 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:58.784309 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4812437-86cb-45ac-be79-611bbb8f8ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:58.784963 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4812437-86cb-45ac-be79-611bbb8f8ec3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:58.785485 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4812437-86cb-45ac-be79-611bbb8f8ec3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:58.787079 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4812437-86cb-45ac-be79-611bbb8f8ec3 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:58.806190 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1044/2077] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:51:58 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:51:58.810265 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c99e54f-218e-48df-b9f3-a8a21be9df5d admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:58.822114 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c99e54f-218e-48df-b9f3-a8a21be9df5d admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:58.829480 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f762d0-4045-4eb6-a5f7-30df982bb90e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:58.830712 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f762d0-4045-4eb6-a5f7-30df982bb90e admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:58.831199 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f762d0-4045-4eb6-a5f7-30df982bb90e admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:58.831490 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f762d0-4045-4eb6-a5f7-30df982bb90e admin admin] PATH_INFO: `/v3/projects/d3579f167dd9458b88b48807d90ab6ad/users/5544312680724764bb1dcc964c1c04a2/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:58.849741 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f762d0-4045-4eb6-a5f7-30df982bb90e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3579f167dd9458b88b48807d90ab6ad, user_id=5544312680724764bb1dcc964c1c04a2, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:58.852532 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f762d0-4045-4eb6-a5f7-30df982bb90e admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:58.957967 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1034/2078] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:51:58 2022] PUT /identity/v3/projects/d3579f167dd9458b88b48807d90ab6ad/users/5544312680724764bb1dcc964c1c04a2/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:58.971863 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb77270-b894-490d-b415-ab790658c97b None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:58.972481 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb77270-b894-490d-b415-ab790658c97b None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:58.972642 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb77270-b894-490d-b415-ab790658c97b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:59.013449 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-8eb77270-b894-490d-b415-ab790658c97b None None] MFA Rules not processed for user `5544312680724764bb1dcc964c1c04a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:59.016020 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-8eb77270-b894-490d-b415-ab790658c97b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:59.086921 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1045/2079] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:51:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:59.091570 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f762d0-4045-4eb6-a5f7-30df982bb90e admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:59.124283 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:59.178006 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f762d0-4045-4eb6-a5f7-30df982bb90e admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:59.185388 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99919b6-ba46-4f48-a8d1-4d7df4a8e14c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:59.186333 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99919b6-ba46-4f48-a8d1-4d7df4a8e14c admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:59.186732 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99919b6-ba46-4f48-a8d1-4d7df4a8e14c admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:59.187137 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99919b6-ba46-4f48-a8d1-4d7df4a8e14c admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:59.188215 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99919b6-ba46-4f48-a8d1-4d7df4a8e14c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:59.190874 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99919b6-ba46-4f48-a8d1-4d7df4a8e14c admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:59.321396 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1035/2080] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:51:59 2022] POST /identity/v3/projects => generated 383 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:51:59.325542 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb77270-b894-490d-b415-ab790658c97b None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:59.337947 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb77270-b894-490d-b415-ab790658c97b None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:59.365844 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:59.397302 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56131aff-6840-4759-8109-c5201b23e7a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:59.398005 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56131aff-6840-4759-8109-c5201b23e7a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:59.398865 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56131aff-6840-4759-8109-c5201b23e7a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:59.399145 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56131aff-6840-4759-8109-c5201b23e7a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:59.399998 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56131aff-6840-4759-8109-c5201b23e7a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:59.402001 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56131aff-6840-4759-8109-c5201b23e7a7 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:59.506318 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1046/2081] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:51:59 2022] POST /identity/v3/users => generated 298 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:51:59.510788 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99919b6-ba46-4f48-a8d1-4d7df4a8e14c admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:59.521610 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99919b6-ba46-4f48-a8d1-4d7df4a8e14c admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:59.529929 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997eeec5-ab2e-4a1b-a33f-1b4c74effe3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:59.530190 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997eeec5-ab2e-4a1b-a33f-1b4c74effe3c admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:59.533018 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997eeec5-ab2e-4a1b-a33f-1b4c74effe3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:59.533018 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997eeec5-ab2e-4a1b-a33f-1b4c74effe3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:59.533018 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997eeec5-ab2e-4a1b-a33f-1b4c74effe3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:59.533308 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997eeec5-ab2e-4a1b-a33f-1b4c74effe3c admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:59.545255 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1036/2082] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:51:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:51:59.549420 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56131aff-6840-4759-8109-c5201b23e7a7 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:59.561952 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56131aff-6840-4759-8109-c5201b23e7a7 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:59.565933 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d7e207-8591-4d5a-a3f0-7166afc0e96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:59.566575 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d7e207-8591-4d5a-a3f0-7166afc0e96e admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:59.566855 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d7e207-8591-4d5a-a3f0-7166afc0e96e admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:59.567160 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d7e207-8591-4d5a-a3f0-7166afc0e96e admin admin] PATH_INFO: `/v3/projects/229ebfe89b9e4adf9cbf0a6547b3d0c9/users/0de68120eddd4255a7b3f991cf657f2f/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:59.582990 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d7e207-8591-4d5a-a3f0-7166afc0e96e admin admin] RBAC: Authorizing `identity:create_grant(project_id=229ebfe89b9e4adf9cbf0a6547b3d0c9, user_id=0de68120eddd4255a7b3f991cf657f2f, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:59.586724 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d7e207-8591-4d5a-a3f0-7166afc0e96e admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:59.632638 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1047/2083] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:51:59 2022] PUT /identity/v3/projects/229ebfe89b9e4adf9cbf0a6547b3d0c9/users/0de68120eddd4255a7b3f991cf657f2f/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:59.637143 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997eeec5-ab2e-4a1b-a33f-1b4c74effe3c admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:59.647238 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997eeec5-ab2e-4a1b-a33f-1b4c74effe3c admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:59.708860 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b471db98-5e58-4d19-83d2-785d476d1d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:59.709538 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b471db98-5e58-4d19-83d2-785d476d1d45 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:59.709821 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b471db98-5e58-4d19-83d2-785d476d1d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:59.710098 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b471db98-5e58-4d19-83d2-785d476d1d45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:59.711338 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b471db98-5e58-4d19-83d2-785d476d1d45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:59.713340 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b471db98-5e58-4d19-83d2-785d476d1d45 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:59.723563 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1037/2084] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:51:59 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:51:59.728839 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d7e207-8591-4d5a-a3f0-7166afc0e96e admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:59.739422 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d7e207-8591-4d5a-a3f0-7166afc0e96e admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:59.744791 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42adc596-397f-4d44-848c-8082abbab0cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:59.745679 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42adc596-397f-4d44-848c-8082abbab0cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:59.745959 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42adc596-397f-4d44-848c-8082abbab0cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:59.746232 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42adc596-397f-4d44-848c-8082abbab0cb admin admin] PATH_INFO: `/v3/projects/229ebfe89b9e4adf9cbf0a6547b3d0c9/users/0de68120eddd4255a7b3f991cf657f2f/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:59.748236 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42adc596-397f-4d44-848c-8082abbab0cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=229ebfe89b9e4adf9cbf0a6547b3d0c9, user_id=0de68120eddd4255a7b3f991cf657f2f, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:59.750865 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42adc596-397f-4d44-848c-8082abbab0cb admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:59.764866 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1048/2085] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:51:59 2022] PUT /identity/v3/projects/229ebfe89b9e4adf9cbf0a6547b3d0c9/users/0de68120eddd4255a7b3f991cf657f2f/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:51:59.774922 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b68cc9-2db7-4255-b90b-f94f7e405c6f None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:59.775209 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b68cc9-2db7-4255-b90b-f94f7e405c6f None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:59.775602 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b68cc9-2db7-4255-b90b-f94f7e405c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:59.806271 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-95b68cc9-2db7-4255-b90b-f94f7e405c6f None None] MFA Rules not processed for user `0de68120eddd4255a7b3f991cf657f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:59.807499 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-95b68cc9-2db7-4255-b90b-f94f7e405c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:59.899129 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1038/2086] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:51:59 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:59.909734 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42adc596-397f-4d44-848c-8082abbab0cb admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:59.911496 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-42adc596-397f-4d44-848c-8082abbab0cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:59.973766 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42adc596-397f-4d44-848c-8082abbab0cb admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:59.977597 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296be17b-9a06-4dcf-9123-ab7a7235ba3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:59.978248 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296be17b-9a06-4dcf-9123-ab7a7235ba3c service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:59.978548 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296be17b-9a06-4dcf-9123-ab7a7235ba3c service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:59.978820 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296be17b-9a06-4dcf-9123-ab7a7235ba3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:59.988888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296be17b-9a06-4dcf-9123-ab7a7235ba3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:59.990338 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296be17b-9a06-4dcf-9123-ab7a7235ba3c service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:00.000908 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1049/2087] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:51:59 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:52:00.126623 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b68cc9-2db7-4255-b90b-f94f7e405c6f None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:00.132990 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:00.137241 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b68cc9-2db7-4255-b90b-f94f7e405c6f None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:00.140800 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af73fb8-eba1-4817-8bbb-851f221a2121 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:00.141451 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af73fb8-eba1-4817-8bbb-851f221a2121 service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:00.141735 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af73fb8-eba1-4817-8bbb-851f221a2121 service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:00.142015 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af73fb8-eba1-4817-8bbb-851f221a2121 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:00.206585 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af73fb8-eba1-4817-8bbb-851f221a2121 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:00.208119 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af73fb8-eba1-4817-8bbb-851f221a2121 service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:00.219457 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1039/2088] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:52:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:52:03.081687 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296be17b-9a06-4dcf-9123-ab7a7235ba3c service cinder] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:03.082583 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-296be17b-9a06-4dcf-9123-ab7a7235ba3c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:03.169839 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296be17b-9a06-4dcf-9123-ab7a7235ba3c service cinder] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:03.174748 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a7baa4-ab59-4ad1-bd1e-b39641f0d03f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:03.175627 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a7baa4-ab59-4ad1-bd1e-b39641f0d03f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:03.175912 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a7baa4-ab59-4ad1-bd1e-b39641f0d03f admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:03.176228 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a7baa4-ab59-4ad1-bd1e-b39641f0d03f admin admin] PATH_INFO: `/v3/users/24ffa9fc64d243dbae86bd4f1fdb7eff` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:03.177419 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a7baa4-ab59-4ad1-bd1e-b39641f0d03f admin admin] RBAC: Authorizing `identity:delete_user(user_id=24ffa9fc64d243dbae86bd4f1fdb7eff)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:03.179349 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a7baa4-ab59-4ad1-bd1e-b39641f0d03f admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:03.358584 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:03.474101 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-a9a7baa4-ab59-4ad1-bd1e-b39641f0d03f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24ffa9fc64d243dbae86bd4f1fdb7eff'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:03.474947 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-a9a7baa4-ab59-4ad1-bd1e-b39641f0d03f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24ffa9fc64d243dbae86bd4f1fdb7eff'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:03.475675 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-a9a7baa4-ab59-4ad1-bd1e-b39641f0d03f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24ffa9fc64d243dbae86bd4f1fdb7eff'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:03.614510 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-a9a7baa4-ab59-4ad1-bd1e-b39641f0d03f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24ffa9fc64d243dbae86bd4f1fdb7eff'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:03.630501 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-a9a7baa4-ab59-4ad1-bd1e-b39641f0d03f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24ffa9fc64d243dbae86bd4f1fdb7eff'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:03.645960 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1050/2089] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:52:03 2022] DELETE /identity/v3/users/24ffa9fc64d243dbae86bd4f1fdb7eff => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:03.651736 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48838afe-322e-4c6b-b8f3-45327bb43904 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:03.652218 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48838afe-322e-4c6b-b8f3-45327bb43904 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:03.652586 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48838afe-322e-4c6b-b8f3-45327bb43904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:03.684895 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-48838afe-322e-4c6b-b8f3-45327bb43904 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:03.686150 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-48838afe-322e-4c6b-b8f3-45327bb43904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:03.745735 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1040/2090] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:52:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:03.755428 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a7baa4-ab59-4ad1-bd1e-b39641f0d03f admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:03.757238 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a9a7baa4-ab59-4ad1-bd1e-b39641f0d03f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:03.827557 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a7baa4-ab59-4ad1-bd1e-b39641f0d03f admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:03.831206 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaef1c3-0f13-4e8c-8bbf-01c17964ce5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:03.831848 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaef1c3-0f13-4e8c-8bbf-01c17964ce5b service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:03.832159 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaef1c3-0f13-4e8c-8bbf-01c17964ce5b service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:03.832438 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaef1c3-0f13-4e8c-8bbf-01c17964ce5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:03.842906 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaef1c3-0f13-4e8c-8bbf-01c17964ce5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:03.845191 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaef1c3-0f13-4e8c-8bbf-01c17964ce5b service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:03.859975 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1051/2091] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:52:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:52:04.622287 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48838afe-322e-4c6b-b8f3-45327bb43904 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:04.623862 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-48838afe-322e-4c6b-b8f3-45327bb43904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:04.638187 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48838afe-322e-4c6b-b8f3-45327bb43904 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:04.643269 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5f28e9-2e43-4a71-bf0b-79e0b6b3511d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:04.644192 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f28e9-2e43-4a71-bf0b-79e0b6b3511d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:04.650085 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f28e9-2e43-4a71-bf0b-79e0b6b3511d admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:04.650085 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f28e9-2e43-4a71-bf0b-79e0b6b3511d admin admin] PATH_INFO: `/v3/projects/a0fa37ad549441d7b51b4463ab2fed53` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:04.650085 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5f28e9-2e43-4a71-bf0b-79e0b6b3511d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0fa37ad549441d7b51b4463ab2fed53)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:04.650085 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5f28e9-2e43-4a71-bf0b-79e0b6b3511d admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:04.926351 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-1b5f28e9-2e43-4a71-bf0b-79e0b6b3511d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0fa37ad549441d7b51b4463ab2fed53', 'request_id': 'req-1b5f28e9-2e43-4a71-bf0b-79e0b6b3511d'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:05.135426 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-1b5f28e9-2e43-4a71-bf0b-79e0b6b3511d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0fa37ad549441d7b51b4463ab2fed53', 'request_id': 'req-1b5f28e9-2e43-4a71-bf0b-79e0b6b3511d'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:05.141892 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-1b5f28e9-2e43-4a71-bf0b-79e0b6b3511d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0fa37ad549441d7b51b4463ab2fed53', 'request_id': 'req-1b5f28e9-2e43-4a71-bf0b-79e0b6b3511d'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:05.153521 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-1b5f28e9-2e43-4a71-bf0b-79e0b6b3511d admin admin] The token cache is being invalidate because project a0fa37ad549441d7b51b4463ab2fed53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:52:05.153807 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-1b5f28e9-2e43-4a71-bf0b-79e0b6b3511d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:05.154466 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-1b5f28e9-2e43-4a71-bf0b-79e0b6b3511d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:05.156044 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1041/2092] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:52:04 2022] DELETE /identity/v3/projects/a0fa37ad549441d7b51b4463ab2fed53 => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:05.160216 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaef1c3-0f13-4e8c-8bbf-01c17964ce5b service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:05.170550 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaef1c3-0f13-4e8c-8bbf-01c17964ce5b service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:05.231509 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1dfaf0-a9d2-4e4e-9f22-0612e6099d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:05.232499 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1dfaf0-a9d2-4e4e-9f22-0612e6099d26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:05.232781 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1dfaf0-a9d2-4e4e-9f22-0612e6099d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:05.233054 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1dfaf0-a9d2-4e4e-9f22-0612e6099d26 admin admin] PATH_INFO: `/v3/users/e1a8c38ef80241fabae4572e11c1da62` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:05.234299 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1dfaf0-a9d2-4e4e-9f22-0612e6099d26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1a8c38ef80241fabae4572e11c1da62)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:05.236325 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1dfaf0-a9d2-4e4e-9f22-0612e6099d26 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:05.558647 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:05.558647 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-aa1dfaf0-a9d2-4e4e-9f22-0612e6099d26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1a8c38ef80241fabae4572e11c1da62'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:05.558647 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-aa1dfaf0-a9d2-4e4e-9f22-0612e6099d26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1a8c38ef80241fabae4572e11c1da62'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:05.558647 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-aa1dfaf0-a9d2-4e4e-9f22-0612e6099d26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1a8c38ef80241fabae4572e11c1da62'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:05.768553 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-aa1dfaf0-a9d2-4e4e-9f22-0612e6099d26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1a8c38ef80241fabae4572e11c1da62'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:05.782468 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-aa1dfaf0-a9d2-4e4e-9f22-0612e6099d26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1a8c38ef80241fabae4572e11c1da62'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:05.798159 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1052/2093] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:52:05 2022] DELETE /identity/v3/users/e1a8c38ef80241fabae4572e11c1da62 => generated 0 bytes in 636 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:06.269201 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5f28e9-2e43-4a71-bf0b-79e0b6b3511d admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:06.271442 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-1b5f28e9-2e43-4a71-bf0b-79e0b6b3511d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:06.290706 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5f28e9-2e43-4a71-bf0b-79e0b6b3511d admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:06.342899 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:06.404809 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b02c11-9055-41ec-8030-abf3c5915e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:06.405899 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b02c11-9055-41ec-8030-abf3c5915e62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:06.406193 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b02c11-9055-41ec-8030-abf3c5915e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:06.407410 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b02c11-9055-41ec-8030-abf3c5915e62 admin admin] PATH_INFO: `/v3/projects/3369fba032fd4fd88704530acfa673d8` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:06.409903 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b02c11-9055-41ec-8030-abf3c5915e62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3369fba032fd4fd88704530acfa673d8)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:06.413742 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b02c11-9055-41ec-8030-abf3c5915e62 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:06.528204 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-82b02c11-9055-41ec-8030-abf3c5915e62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3369fba032fd4fd88704530acfa673d8', 'request_id': 'req-82b02c11-9055-41ec-8030-abf3c5915e62'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:06.648349 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-82b02c11-9055-41ec-8030-abf3c5915e62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3369fba032fd4fd88704530acfa673d8', 'request_id': 'req-82b02c11-9055-41ec-8030-abf3c5915e62'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:06.655669 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-82b02c11-9055-41ec-8030-abf3c5915e62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3369fba032fd4fd88704530acfa673d8', 'request_id': 'req-82b02c11-9055-41ec-8030-abf3c5915e62'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:06.667828 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-82b02c11-9055-41ec-8030-abf3c5915e62 admin admin] The token cache is being invalidate because project 3369fba032fd4fd88704530acfa673d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:52:06.668191 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-82b02c11-9055-41ec-8030-abf3c5915e62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:06.668942 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-82b02c11-9055-41ec-8030-abf3c5915e62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:06.670874 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1042/2094] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:52:06 2022] DELETE /identity/v3/projects/3369fba032fd4fd88704530acfa673d8 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:06.675800 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63396056-5c28-41ea-a7c8-af6ed7ed0e29 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:06.676719 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63396056-5c28-41ea-a7c8-af6ed7ed0e29 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:06.676719 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63396056-5c28-41ea-a7c8-af6ed7ed0e29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:06.717375 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-63396056-5c28-41ea-a7c8-af6ed7ed0e29 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:06.718623 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-63396056-5c28-41ea-a7c8-af6ed7ed0e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:06.787838 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1053/2095] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:52:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:06.789585 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ae3804-b87c-465d-85a9-9b4967c18856 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:06.789787 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ae3804-b87c-465d-85a9-9b4967c18856 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:06.790655 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ae3804-b87c-465d-85a9-9b4967c18856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:06.835800 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-f9ae3804-b87c-465d-85a9-9b4967c18856 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:06.837824 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-f9ae3804-b87c-465d-85a9-9b4967c18856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:06.846115 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1043/2096] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:52:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:52:06.850579 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63396056-5c28-41ea-a7c8-af6ed7ed0e29 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:06.861502 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63396056-5c28-41ea-a7c8-af6ed7ed0e29 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:06.867091 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e60d307-b211-41d5-8f01-dde543bca41b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:06.867836 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e60d307-b211-41d5-8f01-dde543bca41b admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:06.873361 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e60d307-b211-41d5-8f01-dde543bca41b admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:06.873361 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e60d307-b211-41d5-8f01-dde543bca41b admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:06.873361 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e60d307-b211-41d5-8f01-dde543bca41b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:52:06.873361 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e60d307-b211-41d5-8f01-dde543bca41b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:06.873361 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e60d307-b211-41d5-8f01-dde543bca41b admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:06.887479 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1054/2097] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jun 15 14:52:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:52:06.891988 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ae3804-b87c-465d-85a9-9b4967c18856 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:06.902723 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ae3804-b87c-465d-85a9-9b4967c18856 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:06.907499 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779d1cb3-f72b-4b11-8820-67103450df70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:06.908221 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779d1cb3-f72b-4b11-8820-67103450df70 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:06.913514 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779d1cb3-f72b-4b11-8820-67103450df70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:06.913514 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779d1cb3-f72b-4b11-8820-67103450df70 admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:06.913514 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779d1cb3-f72b-4b11-8820-67103450df70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:06.913514 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779d1cb3-f72b-4b11-8820-67103450df70 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:07.027996 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1044/2098] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:52:06 2022] POST /identity/v3/projects => generated 363 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:52:07.036871 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e60d307-b211-41d5-8f01-dde543bca41b admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:07.048321 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e60d307-b211-41d5-8f01-dde543bca41b admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:07.115286 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a338fc-9e74-461a-bbc4-1cc3bf4501cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:07.115971 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a338fc-9e74-461a-bbc4-1cc3bf4501cc admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:07.116284 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a338fc-9e74-461a-bbc4-1cc3bf4501cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:07.116563 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a338fc-9e74-461a-bbc4-1cc3bf4501cc admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:07.117286 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a338fc-9e74-461a-bbc4-1cc3bf4501cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:07.120267 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a338fc-9e74-461a-bbc4-1cc3bf4501cc admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:07.209364 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1055/2099] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:52:07 2022] POST /identity/v3/users => generated 282 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:52:07.213884 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779d1cb3-f72b-4b11-8820-67103450df70 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:07.224450 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779d1cb3-f72b-4b11-8820-67103450df70 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:07.230108 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44a77fe-e2cb-498a-94a4-43e33b2bc95b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:07.230716 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44a77fe-e2cb-498a-94a4-43e33b2bc95b admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:07.230995 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44a77fe-e2cb-498a-94a4-43e33b2bc95b admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:07.231268 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44a77fe-e2cb-498a-94a4-43e33b2bc95b admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:07.234789 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44a77fe-e2cb-498a-94a4-43e33b2bc95b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:07.234789 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44a77fe-e2cb-498a-94a4-43e33b2bc95b admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:07.245789 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1045/2100] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:52:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:52:07.250173 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a338fc-9e74-461a-bbc4-1cc3bf4501cc admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:07.260655 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a338fc-9e74-461a-bbc4-1cc3bf4501cc admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:07.265616 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33269f48-05c7-4182-a445-30b3104aca92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:07.266440 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33269f48-05c7-4182-a445-30b3104aca92 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:07.266722 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33269f48-05c7-4182-a445-30b3104aca92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:07.267001 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33269f48-05c7-4182-a445-30b3104aca92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:07.267731 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33269f48-05c7-4182-a445-30b3104aca92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:07.269356 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33269f48-05c7-4182-a445-30b3104aca92 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:07.280995 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1056/2101] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:52:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:52:07.285289 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44a77fe-e2cb-498a-94a4-43e33b2bc95b admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:07.297420 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44a77fe-e2cb-498a-94a4-43e33b2bc95b admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:07.302663 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8650b2ab-306a-48e4-9cdc-2db962781f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:07.303582 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8650b2ab-306a-48e4-9cdc-2db962781f3a admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:07.303858 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8650b2ab-306a-48e4-9cdc-2db962781f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:07.304219 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8650b2ab-306a-48e4-9cdc-2db962781f3a admin admin] PATH_INFO: `/v3/projects/c93bec6684d64140aa5ed74be99a94a1/users/bdcf2dcc1b8a418faab18e40f4e2a598/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:07.319935 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8650b2ab-306a-48e4-9cdc-2db962781f3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c93bec6684d64140aa5ed74be99a94a1, user_id=bdcf2dcc1b8a418faab18e40f4e2a598, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:07.323014 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8650b2ab-306a-48e4-9cdc-2db962781f3a admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:07.410844 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1046/2102] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:52:07 2022] PUT /identity/v3/projects/c93bec6684d64140aa5ed74be99a94a1/users/bdcf2dcc1b8a418faab18e40f4e2a598/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:07.428375 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e622bef1-0dc5-4c86-bd8c-0c55af9d5e53 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:07.428691 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e622bef1-0dc5-4c86-bd8c-0c55af9d5e53 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:07.429286 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e622bef1-0dc5-4c86-bd8c-0c55af9d5e53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:07.443084 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:07.459321 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-e622bef1-0dc5-4c86-bd8c-0c55af9d5e53 None None] MFA Rules not processed for user `bdcf2dcc1b8a418faab18e40f4e2a598`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:07.460925 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e622bef1-0dc5-4c86-bd8c-0c55af9d5e53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:07.548779 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1057/2103] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:52:07 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:07.553036 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8650b2ab-306a-48e4-9cdc-2db962781f3a admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:07.613943 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:07.629392 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8650b2ab-306a-48e4-9cdc-2db962781f3a admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:07.635216 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f544d-897d-484c-838c-589f034d8a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:07.635909 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f544d-897d-484c-838c-589f034d8a21 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:07.636511 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f544d-897d-484c-838c-589f034d8a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:07.636790 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f544d-897d-484c-838c-589f034d8a21 admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:07.637489 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5f544d-897d-484c-838c-589f034d8a21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:07.639997 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5f544d-897d-484c-838c-589f034d8a21 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:07.811445 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1047/2104] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:52:07 2022] POST /identity/v3/projects => generated 365 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:52:07.816790 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e622bef1-0dc5-4c86-bd8c-0c55af9d5e53 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:07.838703 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e622bef1-0dc5-4c86-bd8c-0c55af9d5e53 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:07.911382 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b986fe-dc70-44ea-ac40-6e8321bb26ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:07.912073 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b986fe-dc70-44ea-ac40-6e8321bb26ae admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:07.912410 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b986fe-dc70-44ea-ac40-6e8321bb26ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:07.912707 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b986fe-dc70-44ea-ac40-6e8321bb26ae admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:07.913559 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b986fe-dc70-44ea-ac40-6e8321bb26ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:07.915875 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b986fe-dc70-44ea-ac40-6e8321bb26ae admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:08.081915 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1058/2105] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:52:07 2022] POST /identity/v3/users => generated 289 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:52:08.083285 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f544d-897d-484c-838c-589f034d8a21 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:08.094134 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f544d-897d-484c-838c-589f034d8a21 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:08.099405 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0103ae7-2114-47d3-a42a-7047dab2ef51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:08.100117 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0103ae7-2114-47d3-a42a-7047dab2ef51 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:08.100408 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0103ae7-2114-47d3-a42a-7047dab2ef51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:08.100697 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0103ae7-2114-47d3-a42a-7047dab2ef51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:08.101460 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0103ae7-2114-47d3-a42a-7047dab2ef51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:08.103215 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0103ae7-2114-47d3-a42a-7047dab2ef51 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:08.115013 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1048/2106] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:52:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:52:08.120109 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b986fe-dc70-44ea-ac40-6e8321bb26ae admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:08.130993 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b986fe-dc70-44ea-ac40-6e8321bb26ae admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:08.138449 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2e8157-8538-4ded-8d2e-6aa848c82240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:08.139485 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e8157-8538-4ded-8d2e-6aa848c82240 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:08.139692 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e8157-8538-4ded-8d2e-6aa848c82240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:08.139947 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e8157-8538-4ded-8d2e-6aa848c82240 admin admin] PATH_INFO: `/v3/projects/a62890f2eb7c4ecc8b0af3d527c65754/users/0b69191274ef4538981a8d0013c3b8b8/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:08.165482 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2e8157-8538-4ded-8d2e-6aa848c82240 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a62890f2eb7c4ecc8b0af3d527c65754, user_id=0b69191274ef4538981a8d0013c3b8b8, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:08.166476 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2e8157-8538-4ded-8d2e-6aa848c82240 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:08.262382 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1059/2107] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:52:08 2022] PUT /identity/v3/projects/a62890f2eb7c4ecc8b0af3d527c65754/users/0b69191274ef4538981a8d0013c3b8b8/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:08.266643 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0103ae7-2114-47d3-a42a-7047dab2ef51 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:08.278850 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0103ae7-2114-47d3-a42a-7047dab2ef51 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:08.339196 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2879200d-4363-4ec0-980f-aa4aa84062e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:08.339901 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2879200d-4363-4ec0-980f-aa4aa84062e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:08.340165 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2879200d-4363-4ec0-980f-aa4aa84062e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:08.340445 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2879200d-4363-4ec0-980f-aa4aa84062e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:08.341168 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2879200d-4363-4ec0-980f-aa4aa84062e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:08.342880 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2879200d-4363-4ec0-980f-aa4aa84062e3 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:08.357593 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1049/2108] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:52:08 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:52:08.361448 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2e8157-8538-4ded-8d2e-6aa848c82240 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:08.372960 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2e8157-8538-4ded-8d2e-6aa848c82240 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:08.377634 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26e91da-f548-4170-9b73-be943fedcd91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:08.378520 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26e91da-f548-4170-9b73-be943fedcd91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:08.378799 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26e91da-f548-4170-9b73-be943fedcd91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:08.379076 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26e91da-f548-4170-9b73-be943fedcd91 admin admin] PATH_INFO: `/v3/projects/a62890f2eb7c4ecc8b0af3d527c65754/users/0b69191274ef4538981a8d0013c3b8b8/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:08.381087 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26e91da-f548-4170-9b73-be943fedcd91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a62890f2eb7c4ecc8b0af3d527c65754, user_id=0b69191274ef4538981a8d0013c3b8b8, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:08.386197 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26e91da-f548-4170-9b73-be943fedcd91 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:08.399022 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1060/2109] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:52:08 2022] PUT /identity/v3/projects/a62890f2eb7c4ecc8b0af3d527c65754/users/0b69191274ef4538981a8d0013c3b8b8/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:08.410860 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c42398-0bc4-4042-830c-1acf11ec4759 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:08.412858 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c42398-0bc4-4042-830c-1acf11ec4759 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:08.412858 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c42398-0bc4-4042-830c-1acf11ec4759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:08.444364 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-d6c42398-0bc4-4042-830c-1acf11ec4759 None None] MFA Rules not processed for user `0b69191274ef4538981a8d0013c3b8b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:08.446039 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-d6c42398-0bc4-4042-830c-1acf11ec4759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:08.521672 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1050/2110] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:52:08 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:08.533962 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26e91da-f548-4170-9b73-be943fedcd91 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:08.541697 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e26e91da-f548-4170-9b73-be943fedcd91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:08.617333 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:08.622348 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26e91da-f548-4170-9b73-be943fedcd91 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:08.626039 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396a4462-eb00-42a3-8356-4c345c2fcc28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:08.626688 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396a4462-eb00-42a3-8356-4c345c2fcc28 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:08.626994 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396a4462-eb00-42a3-8356-4c345c2fcc28 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:08.627251 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396a4462-eb00-42a3-8356-4c345c2fcc28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:08.637730 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396a4462-eb00-42a3-8356-4c345c2fcc28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:08.639186 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396a4462-eb00-42a3-8356-4c345c2fcc28 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:08.654121 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1061/2111] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:52:08 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:52:08.954441 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c42398-0bc4-4042-830c-1acf11ec4759 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:08.956430 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-d6c42398-0bc4-4042-830c-1acf11ec4759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:09.019089 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:09.046244 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c42398-0bc4-4042-830c-1acf11ec4759 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:09.051277 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6075ba0-f517-4afc-b4e6-6b3a03186175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:09.057049 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6075ba0-f517-4afc-b4e6-6b3a03186175 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:09.057049 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6075ba0-f517-4afc-b4e6-6b3a03186175 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:09.057049 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6075ba0-f517-4afc-b4e6-6b3a03186175 admin admin] PATH_INFO: `/v3/users/5544312680724764bb1dcc964c1c04a2` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:09.057049 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6075ba0-f517-4afc-b4e6-6b3a03186175 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5544312680724764bb1dcc964c1c04a2)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:09.057049 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6075ba0-f517-4afc-b4e6-6b3a03186175 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:09.342023 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-d6075ba0-f517-4afc-b4e6-6b3a03186175 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5544312680724764bb1dcc964c1c04a2'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:09.342023 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-d6075ba0-f517-4afc-b4e6-6b3a03186175 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5544312680724764bb1dcc964c1c04a2'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:09.342023 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-d6075ba0-f517-4afc-b4e6-6b3a03186175 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5544312680724764bb1dcc964c1c04a2'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:09.342023 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-d6075ba0-f517-4afc-b4e6-6b3a03186175 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5544312680724764bb1dcc964c1c04a2'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:09.465570 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-d6075ba0-f517-4afc-b4e6-6b3a03186175 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5544312680724764bb1dcc964c1c04a2'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:09.491797 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1051/2112] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:52:08 2022] DELETE /identity/v3/users/5544312680724764bb1dcc964c1c04a2 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:09.497406 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23528290-8c18-4f89-be84-9f5308e0e6bb None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:09.497868 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23528290-8c18-4f89-be84-9f5308e0e6bb None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:09.498072 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23528290-8c18-4f89-be84-9f5308e0e6bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:09.516397 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-23528290-8c18-4f89-be84-9f5308e0e6bb None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:09.517609 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-23528290-8c18-4f89-be84-9f5308e0e6bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:09.609745 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1062/2113] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:52:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:09.620447 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6075ba0-f517-4afc-b4e6-6b3a03186175 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:09.621841 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-d6075ba0-f517-4afc-b4e6-6b3a03186175 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:09.681424 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6075ba0-f517-4afc-b4e6-6b3a03186175 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:09.685034 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b24dde-991b-4e18-920d-0ff6c3e00479 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:09.685676 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b24dde-991b-4e18-920d-0ff6c3e00479 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:09.685957 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b24dde-991b-4e18-920d-0ff6c3e00479 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:09.686237 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b24dde-991b-4e18-920d-0ff6c3e00479 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:09.695358 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b24dde-991b-4e18-920d-0ff6c3e00479 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:09.696885 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b24dde-991b-4e18-920d-0ff6c3e00479 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:09.706989 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1052/2114] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:52:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:52:09.866785 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23528290-8c18-4f89-be84-9f5308e0e6bb None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:09.868617 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-23528290-8c18-4f89-be84-9f5308e0e6bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:09.881806 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23528290-8c18-4f89-be84-9f5308e0e6bb None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:09.887420 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8347d3b-afe8-4592-9329-009cc9c89e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:09.888368 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8347d3b-afe8-4592-9329-009cc9c89e1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:09.888648 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8347d3b-afe8-4592-9329-009cc9c89e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:09.888923 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8347d3b-afe8-4592-9329-009cc9c89e1e admin admin] PATH_INFO: `/v3/users/bdcf2dcc1b8a418faab18e40f4e2a598` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:09.890135 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8347d3b-afe8-4592-9329-009cc9c89e1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdcf2dcc1b8a418faab18e40f4e2a598)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:09.892072 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8347d3b-afe8-4592-9329-009cc9c89e1e admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:10.133129 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-f8347d3b-afe8-4592-9329-009cc9c89e1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdcf2dcc1b8a418faab18e40f4e2a598'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:10.135008 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-f8347d3b-afe8-4592-9329-009cc9c89e1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdcf2dcc1b8a418faab18e40f4e2a598'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:10.136365 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-f8347d3b-afe8-4592-9329-009cc9c89e1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdcf2dcc1b8a418faab18e40f4e2a598'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:10.217286 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b24dde-991b-4e18-920d-0ff6c3e00479 service neutron] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:10.222299 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-18b24dde-991b-4e18-920d-0ff6c3e00479 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:10.234981 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b24dde-991b-4e18-920d-0ff6c3e00479 service neutron] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:10.240300 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9051dbc2-4000-43b7-ae97-04110b68c0ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:10.241176 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9051dbc2-4000-43b7-ae97-04110b68c0ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:10.241825 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9051dbc2-4000-43b7-ae97-04110b68c0ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:10.242115 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9051dbc2-4000-43b7-ae97-04110b68c0ed admin admin] PATH_INFO: `/v3/projects/d3579f167dd9458b88b48807d90ab6ad` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:10.243359 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9051dbc2-4000-43b7-ae97-04110b68c0ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3579f167dd9458b88b48807d90ab6ad)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:10.243547 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-f8347d3b-afe8-4592-9329-009cc9c89e1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdcf2dcc1b8a418faab18e40f4e2a598'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:10.246037 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9051dbc2-4000-43b7-ae97-04110b68c0ed admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:10.265373 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-f8347d3b-afe8-4592-9329-009cc9c89e1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdcf2dcc1b8a418faab18e40f4e2a598'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:10.286235 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1063/2115] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:52:09 2022] DELETE /identity/v3/users/bdcf2dcc1b8a418faab18e40f4e2a598 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:10.297324 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f1d0ba-7696-4b83-b7f0-b1a75e577e78 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:10.297708 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f1d0ba-7696-4b83-b7f0-b1a75e577e78 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:10.297947 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f1d0ba-7696-4b83-b7f0-b1a75e577e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:10.318681 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-52f1d0ba-7696-4b83-b7f0-b1a75e577e78 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:10.319955 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-52f1d0ba-7696-4b83-b7f0-b1a75e577e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:10.391237 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1064/2116] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:52:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:10.401994 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f1d0ba-7696-4b83-b7f0-b1a75e577e78 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:10.404310 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-52f1d0ba-7696-4b83-b7f0-b1a75e577e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:10.419195 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:10.420537 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-9051dbc2-4000-43b7-ae97-04110b68c0ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3579f167dd9458b88b48807d90ab6ad', 'request_id': 'req-9051dbc2-4000-43b7-ae97-04110b68c0ed'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:10.481231 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f1d0ba-7696-4b83-b7f0-b1a75e577e78 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:10.532927 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc22ba1-15a5-4326-a5eb-53ce740a0927 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:10.533635 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc22ba1-15a5-4326-a5eb-53ce740a0927 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:10.533857 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc22ba1-15a5-4326-a5eb-53ce740a0927 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:10.534307 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc22ba1-15a5-4326-a5eb-53ce740a0927 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:10.535605 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-9051dbc2-4000-43b7-ae97-04110b68c0ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3579f167dd9458b88b48807d90ab6ad', 'request_id': 'req-9051dbc2-4000-43b7-ae97-04110b68c0ed'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:10.538489 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:10.548506 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-9051dbc2-4000-43b7-ae97-04110b68c0ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3579f167dd9458b88b48807d90ab6ad', 'request_id': 'req-9051dbc2-4000-43b7-ae97-04110b68c0ed'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:10.562824 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-9051dbc2-4000-43b7-ae97-04110b68c0ed admin admin] The token cache is being invalidate because project d3579f167dd9458b88b48807d90ab6ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:52:10.563135 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-9051dbc2-4000-43b7-ae97-04110b68c0ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:10.564253 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-9051dbc2-4000-43b7-ae97-04110b68c0ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:10.566753 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1053/2117] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:52:10 2022] DELETE /identity/v3/projects/d3579f167dd9458b88b48807d90ab6ad => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:52:10.570921 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9051dbc2-4000-43b7-ae97-04110b68c0ed admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:10.572605 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-9051dbc2-4000-43b7-ae97-04110b68c0ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:10.604077 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc22ba1-15a5-4326-a5eb-53ce740a0927 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:10.605646 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc22ba1-15a5-4326-a5eb-53ce740a0927 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:10.615945 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1065/2118] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:52:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:52:10.644818 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9051dbc2-4000-43b7-ae97-04110b68c0ed admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:10.650134 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78586cd-30a4-4f80-9998-56d6a0bd8290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:10.651058 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78586cd-30a4-4f80-9998-56d6a0bd8290 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:10.651339 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78586cd-30a4-4f80-9998-56d6a0bd8290 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:10.651669 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78586cd-30a4-4f80-9998-56d6a0bd8290 admin admin] PATH_INFO: `/v3/users/0de68120eddd4255a7b3f991cf657f2f` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:10.652887 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78586cd-30a4-4f80-9998-56d6a0bd8290 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0de68120eddd4255a7b3f991cf657f2f)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:10.654875 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78586cd-30a4-4f80-9998-56d6a0bd8290 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:10.926651 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-b78586cd-30a4-4f80-9998-56d6a0bd8290 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0de68120eddd4255a7b3f991cf657f2f'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:10.927374 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-b78586cd-30a4-4f80-9998-56d6a0bd8290 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0de68120eddd4255a7b3f991cf657f2f'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:10.937200 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-b78586cd-30a4-4f80-9998-56d6a0bd8290 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0de68120eddd4255a7b3f991cf657f2f'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:10.938069 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-b78586cd-30a4-4f80-9998-56d6a0bd8290 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0de68120eddd4255a7b3f991cf657f2f'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:11.100081 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-b78586cd-30a4-4f80-9998-56d6a0bd8290 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0de68120eddd4255a7b3f991cf657f2f'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:11.120327 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1054/2119] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:52:10 2022] DELETE /identity/v3/users/0de68120eddd4255a7b3f991cf657f2f => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:52:11.218843 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc22ba1-15a5-4326-a5eb-53ce740a0927 service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:11.220372 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-2fc22ba1-15a5-4326-a5eb-53ce740a0927 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:11.233240 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc22ba1-15a5-4326-a5eb-53ce740a0927 service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:11.311367 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586d19f8-afce-4208-838a-5fd940c2a832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:11.312308 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586d19f8-afce-4208-838a-5fd940c2a832 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:11.312601 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586d19f8-afce-4208-838a-5fd940c2a832 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:11.312885 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586d19f8-afce-4208-838a-5fd940c2a832 admin admin] PATH_INFO: `/v3/projects/c93bec6684d64140aa5ed74be99a94a1` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:11.314141 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586d19f8-afce-4208-838a-5fd940c2a832 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c93bec6684d64140aa5ed74be99a94a1)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:11.316219 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586d19f8-afce-4208-838a-5fd940c2a832 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:11.329811 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:11.446875 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-586d19f8-afce-4208-838a-5fd940c2a832 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c93bec6684d64140aa5ed74be99a94a1', 'request_id': 'req-586d19f8-afce-4208-838a-5fd940c2a832'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:11.454471 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-586d19f8-afce-4208-838a-5fd940c2a832 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c93bec6684d64140aa5ed74be99a94a1', 'request_id': 'req-586d19f8-afce-4208-838a-5fd940c2a832'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:11.534733 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-586d19f8-afce-4208-838a-5fd940c2a832 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c93bec6684d64140aa5ed74be99a94a1', 'request_id': 'req-586d19f8-afce-4208-838a-5fd940c2a832'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:11.554106 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-586d19f8-afce-4208-838a-5fd940c2a832 admin admin] The token cache is being invalidate because project c93bec6684d64140aa5ed74be99a94a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:52:11.554689 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-586d19f8-afce-4208-838a-5fd940c2a832 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:11.555643 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-586d19f8-afce-4208-838a-5fd940c2a832 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:11.559203 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1066/2120] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:52:11 2022] DELETE /identity/v3/projects/c93bec6684d64140aa5ed74be99a94a1 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:52:11.563231 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78586cd-30a4-4f80-9998-56d6a0bd8290 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:11.564852 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-b78586cd-30a4-4f80-9998-56d6a0bd8290 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:11.579557 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78586cd-30a4-4f80-9998-56d6a0bd8290 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:11.640883 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6f0e5c-39f4-4d36-9490-38b4192a41f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:11.641772 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f0e5c-39f4-4d36-9490-38b4192a41f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:11.642053 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f0e5c-39f4-4d36-9490-38b4192a41f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:11.642327 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f0e5c-39f4-4d36-9490-38b4192a41f7 admin admin] PATH_INFO: `/v3/users/0b69191274ef4538981a8d0013c3b8b8` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:11.643571 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6f0e5c-39f4-4d36-9490-38b4192a41f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b69191274ef4538981a8d0013c3b8b8)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:11.645572 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6f0e5c-39f4-4d36-9490-38b4192a41f7 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:11.653783 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:11.765653 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586d19f8-afce-4208-838a-5fd940c2a832 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:11.767470 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-586d19f8-afce-4208-838a-5fd940c2a832 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:11.781957 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586d19f8-afce-4208-838a-5fd940c2a832 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:11.787266 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a68807c-bc74-43c6-a4b6-2f131f5dacbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:11.788155 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a68807c-bc74-43c6-a4b6-2f131f5dacbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:11.788563 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a68807c-bc74-43c6-a4b6-2f131f5dacbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:11.788949 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a68807c-bc74-43c6-a4b6-2f131f5dacbe admin admin] PATH_INFO: `/v3/projects/229ebfe89b9e4adf9cbf0a6547b3d0c9` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:11.790949 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a68807c-bc74-43c6-a4b6-2f131f5dacbe admin admin] RBAC: Authorizing `identity:delete_project(project_id=229ebfe89b9e4adf9cbf0a6547b3d0c9)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:11.792924 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a68807c-bc74-43c6-a4b6-2f131f5dacbe admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:11.847294 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-4d6f0e5c-39f4-4d36-9490-38b4192a41f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b69191274ef4538981a8d0013c3b8b8'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:11.847978 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-4d6f0e5c-39f4-4d36-9490-38b4192a41f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b69191274ef4538981a8d0013c3b8b8'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:11.856516 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-4d6f0e5c-39f4-4d36-9490-38b4192a41f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b69191274ef4538981a8d0013c3b8b8'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:11.857192 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-4d6f0e5c-39f4-4d36-9490-38b4192a41f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b69191274ef4538981a8d0013c3b8b8'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:11.962639 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-4d6f0e5c-39f4-4d36-9490-38b4192a41f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b69191274ef4538981a8d0013c3b8b8'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:11.984834 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1055/2121] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:52:11 2022] DELETE /identity/v3/users/0b69191274ef4538981a8d0013c3b8b8 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:11.991089 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-0a68807c-bc74-43c6-a4b6-2f131f5dacbe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '229ebfe89b9e4adf9cbf0a6547b3d0c9', 'request_id': 'req-0a68807c-bc74-43c6-a4b6-2f131f5dacbe'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:11.997879 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-0a68807c-bc74-43c6-a4b6-2f131f5dacbe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '229ebfe89b9e4adf9cbf0a6547b3d0c9', 'request_id': 'req-0a68807c-bc74-43c6-a4b6-2f131f5dacbe'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:12.094703 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-0a68807c-bc74-43c6-a4b6-2f131f5dacbe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '229ebfe89b9e4adf9cbf0a6547b3d0c9', 'request_id': 'req-0a68807c-bc74-43c6-a4b6-2f131f5dacbe'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:12.106452 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-0a68807c-bc74-43c6-a4b6-2f131f5dacbe admin admin] The token cache is being invalidate because project 229ebfe89b9e4adf9cbf0a6547b3d0c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:52:12.106739 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-0a68807c-bc74-43c6-a4b6-2f131f5dacbe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:12.107367 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-0a68807c-bc74-43c6-a4b6-2f131f5dacbe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:12.108969 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1067/2122] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:52:11 2022] DELETE /identity/v3/projects/229ebfe89b9e4adf9cbf0a6547b3d0c9 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:12.113787 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad87b3c-169f-4c92-9e16-1bab2c86700e None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:12.114147 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad87b3c-169f-4c92-9e16-1bab2c86700e None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:12.114488 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad87b3c-169f-4c92-9e16-1bab2c86700e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:12.143184 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-1ad87b3c-169f-4c92-9e16-1bab2c86700e None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:12.144414 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-1ad87b3c-169f-4c92-9e16-1bab2c86700e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:12.206599 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1056/2123] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:52:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:12.216936 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9da07f-c5bd-4c37-a442-7eeec4ba34be None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:12.217277 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9da07f-c5bd-4c37-a442-7eeec4ba34be None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:12.217707 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9da07f-c5bd-4c37-a442-7eeec4ba34be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:12.237910 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-0d9da07f-c5bd-4c37-a442-7eeec4ba34be None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:12.238885 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-0d9da07f-c5bd-4c37-a442-7eeec4ba34be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:12.247977 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1068/2124] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:52:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:12.252260 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad87b3c-169f-4c92-9e16-1bab2c86700e None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:12.262766 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad87b3c-169f-4c92-9e16-1bab2c86700e None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:12.267466 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248545f4-0532-4563-9001-f8fc9c6bdcf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:12.268153 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248545f4-0532-4563-9001-f8fc9c6bdcf7 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:12.268435 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248545f4-0532-4563-9001-f8fc9c6bdcf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:12.268713 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248545f4-0532-4563-9001-f8fc9c6bdcf7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:12.269440 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248545f4-0532-4563-9001-f8fc9c6bdcf7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:52:12.269795 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248545f4-0532-4563-9001-f8fc9c6bdcf7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:12.271426 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248545f4-0532-4563-9001-f8fc9c6bdcf7 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:12.293022 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1057/2125] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jun 15 14:52:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:52:12.297501 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9da07f-c5bd-4c37-a442-7eeec4ba34be None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:12.307536 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9da07f-c5bd-4c37-a442-7eeec4ba34be None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:12.312287 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee7df22-fc03-4afc-85fd-b6d083e10d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:12.312949 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee7df22-fc03-4afc-85fd-b6d083e10d00 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:12.313238 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee7df22-fc03-4afc-85fd-b6d083e10d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:12.313552 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee7df22-fc03-4afc-85fd-b6d083e10d00 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:12.314340 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee7df22-fc03-4afc-85fd-b6d083e10d00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:12.316401 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee7df22-fc03-4afc-85fd-b6d083e10d00 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:12.473325 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1069/2126] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:52:12 2022] POST /identity/v3/projects => generated 409 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:52:12.477597 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248545f4-0532-4563-9001-f8fc9c6bdcf7 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:12.487837 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248545f4-0532-4563-9001-f8fc9c6bdcf7 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:12.496568 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee7df22-fc03-4afc-85fd-b6d083e10d00 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:12.501169 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-6ee7df22-fc03-4afc-85fd-b6d083e10d00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:12.572149 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43f7ea3-e894-438d-ba23-915949ef930d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:12.573149 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f7ea3-e894-438d-ba23-915949ef930d admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:12.573435 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f7ea3-e894-438d-ba23-915949ef930d admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:12.573712 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f7ea3-e894-438d-ba23-915949ef930d admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:12.574412 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43f7ea3-e894-438d-ba23-915949ef930d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:12.576391 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43f7ea3-e894-438d-ba23-915949ef930d admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:12.588970 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:12.594534 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:12.599633 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee7df22-fc03-4afc-85fd-b6d083e10d00 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:12.605669 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14200c4-34de-4024-85d8-83294a5a8181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:12.606588 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14200c4-34de-4024-85d8-83294a5a8181 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:12.606860 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14200c4-34de-4024-85d8-83294a5a8181 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:12.607139 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14200c4-34de-4024-85d8-83294a5a8181 admin admin] PATH_INFO: `/v3/projects/a62890f2eb7c4ecc8b0af3d527c65754` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:12.608397 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14200c4-34de-4024-85d8-83294a5a8181 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a62890f2eb7c4ecc8b0af3d527c65754)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:12.610492 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14200c4-34de-4024-85d8-83294a5a8181 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:12.751749 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1058/2127] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:52:12 2022] POST /identity/v3/users => generated 305 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:52:12.755673 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43f7ea3-e894-438d-ba23-915949ef930d admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:12.766771 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43f7ea3-e894-438d-ba23-915949ef930d admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:12.771755 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1452ffbb-f50d-4035-b849-db212445b1e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:12.772543 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1452ffbb-f50d-4035-b849-db212445b1e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:12.772834 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1452ffbb-f50d-4035-b849-db212445b1e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:12.773118 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1452ffbb-f50d-4035-b849-db212445b1e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:12.773848 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1452ffbb-f50d-4035-b849-db212445b1e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:12.776647 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1452ffbb-f50d-4035-b849-db212445b1e8 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:12.791798 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1059/2128] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:52:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:52:12.791999 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-d14200c4-34de-4024-85d8-83294a5a8181 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a62890f2eb7c4ecc8b0af3d527c65754', 'request_id': 'req-d14200c4-34de-4024-85d8-83294a5a8181'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:12.793120 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1452ffbb-f50d-4035-b849-db212445b1e8 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:12.796374 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-d14200c4-34de-4024-85d8-83294a5a8181 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a62890f2eb7c4ecc8b0af3d527c65754', 'request_id': 'req-d14200c4-34de-4024-85d8-83294a5a8181'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:12.803301 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1452ffbb-f50d-4035-b849-db212445b1e8 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:12.861302 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ac3362-cc25-4893-8025-9cbd9fe09919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:12.861924 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac3362-cc25-4893-8025-9cbd9fe09919 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:12.862206 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac3362-cc25-4893-8025-9cbd9fe09919 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:12.862480 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac3362-cc25-4893-8025-9cbd9fe09919 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:12.863197 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ac3362-cc25-4893-8025-9cbd9fe09919 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:12.864847 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ac3362-cc25-4893-8025-9cbd9fe09919 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:12.874997 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1060/2129] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:52:12 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:52:12.878981 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ac3362-cc25-4893-8025-9cbd9fe09919 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:12.889572 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ac3362-cc25-4893-8025-9cbd9fe09919 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:12.894359 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c791d6d-ae0b-4bf8-bfa0-27c544fa689d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:12.895208 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c791d6d-ae0b-4bf8-bfa0-27c544fa689d admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:12.895485 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c791d6d-ae0b-4bf8-bfa0-27c544fa689d admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:12.895758 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c791d6d-ae0b-4bf8-bfa0-27c544fa689d admin admin] PATH_INFO: `/v3/projects/a105d0e9d4074f2ba1c131df2a5a9fe9/users/6cdea625d3a74f2babd149314df03cb7/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:12.913054 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c791d6d-ae0b-4bf8-bfa0-27c544fa689d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a105d0e9d4074f2ba1c131df2a5a9fe9, user_id=6cdea625d3a74f2babd149314df03cb7, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:12.918748 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c791d6d-ae0b-4bf8-bfa0-27c544fa689d admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:12.956051 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-d14200c4-34de-4024-85d8-83294a5a8181 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a62890f2eb7c4ecc8b0af3d527c65754', 'request_id': 'req-d14200c4-34de-4024-85d8-83294a5a8181'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:12.970127 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-d14200c4-34de-4024-85d8-83294a5a8181 admin admin] The token cache is being invalidate because project a62890f2eb7c4ecc8b0af3d527c65754 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:52:12.971204 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-d14200c4-34de-4024-85d8-83294a5a8181 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:12.972306 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-d14200c4-34de-4024-85d8-83294a5a8181 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:12.973935 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1070/2130] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:52:12 2022] DELETE /identity/v3/projects/a62890f2eb7c4ecc8b0af3d527c65754 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:52:12.980703 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09b8cb-4306-488a-af7a-eec01c1e2225 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:12.980703 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09b8cb-4306-488a-af7a-eec01c1e2225 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:12.981028 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09b8cb-4306-488a-af7a-eec01c1e2225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:13.000856 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-bb09b8cb-4306-488a-af7a-eec01c1e2225 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:13.002071 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-bb09b8cb-4306-488a-af7a-eec01c1e2225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:13.010574 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1071/2131] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:52:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:52:13.021251 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e2d7e-dad4-427e-967e-ebd295976864 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:13.021537 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e2d7e-dad4-427e-967e-ebd295976864 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:13.021843 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e2d7e-dad4-427e-967e-ebd295976864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:13.040464 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-627e2d7e-dad4-427e-967e-ebd295976864 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:13.041658 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-627e2d7e-dad4-427e-967e-ebd295976864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:13.049457 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1072/2132] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:52:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:52:13.053505 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627e2d7e-dad4-427e-967e-ebd295976864 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:13.062469 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627e2d7e-dad4-427e-967e-ebd295976864 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:13.066957 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dc6e4c-3fac-4629-b60a-69887e3d3ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:13.067578 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dc6e4c-3fac-4629-b60a-69887e3d3ab3 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:13.067856 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dc6e4c-3fac-4629-b60a-69887e3d3ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:13.068225 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dc6e4c-3fac-4629-b60a-69887e3d3ab3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:13.068941 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dc6e4c-3fac-4629-b60a-69887e3d3ab3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:52:13.069247 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dc6e4c-3fac-4629-b60a-69887e3d3ab3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:13.070820 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dc6e4c-3fac-4629-b60a-69887e3d3ab3 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:13.082974 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1061/2133] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:52:12 2022] PUT /identity/v3/projects/a105d0e9d4074f2ba1c131df2a5a9fe9/users/6cdea625d3a74f2babd149314df03cb7/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:52:13.086537 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1073/2134] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jun 15 14:52:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:52:13.090394 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c791d6d-ae0b-4bf8-bfa0-27c544fa689d admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:13.096111 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e9112d-9185-480c-b58a-acbe68f1952a None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:13.096427 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e9112d-9185-480c-b58a-acbe68f1952a None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:13.096704 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e9112d-9185-480c-b58a-acbe68f1952a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:13.123475 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-b3e9112d-9185-480c-b58a-acbe68f1952a None None] MFA Rules not processed for user `6cdea625d3a74f2babd149314df03cb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:13.124666 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-b3e9112d-9185-480c-b58a-acbe68f1952a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:13.155302 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c791d6d-ae0b-4bf8-bfa0-27c544fa689d admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:13.160061 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0790cca-66e7-4cc4-881a-be0f94d845dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:13.160743 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0790cca-66e7-4cc4-881a-be0f94d845dc admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:13.161029 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0790cca-66e7-4cc4-881a-be0f94d845dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:13.161349 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0790cca-66e7-4cc4-881a-be0f94d845dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:13.162048 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0790cca-66e7-4cc4-881a-be0f94d845dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:13.163935 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0790cca-66e7-4cc4-881a-be0f94d845dc admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:13.197739 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1074/2135] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:52:13 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:52:13.202149 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e9112d-9185-480c-b58a-acbe68f1952a None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:13.205432 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-b3e9112d-9185-480c-b58a-acbe68f1952a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:13.221385 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e9112d-9185-480c-b58a-acbe68f1952a None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:13.226169 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c52b57-d397-45bf-a32d-e52c7ef58bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:13.226845 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c52b57-d397-45bf-a32d-e52c7ef58bcd admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:13.227132 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c52b57-d397-45bf-a32d-e52c7ef58bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:13.227414 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c52b57-d397-45bf-a32d-e52c7ef58bcd admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:13.233462 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c52b57-d397-45bf-a32d-e52c7ef58bcd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:13.233462 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c52b57-d397-45bf-a32d-e52c7ef58bcd admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:13.268293 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1062/2136] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:52:13 2022] POST /identity/v3/projects => generated 385 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:52:13.272603 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0790cca-66e7-4cc4-881a-be0f94d845dc admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:13.315660 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:13.335464 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0790cca-66e7-4cc4-881a-be0f94d845dc admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:13.340356 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351b0314-1213-4d51-ac31-3bce2368d60f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:13.341048 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351b0314-1213-4d51-ac31-3bce2368d60f admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:13.341331 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351b0314-1213-4d51-ac31-3bce2368d60f admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:13.341606 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351b0314-1213-4d51-ac31-3bce2368d60f admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:13.342285 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351b0314-1213-4d51-ac31-3bce2368d60f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:13.354309 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351b0314-1213-4d51-ac31-3bce2368d60f admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:13.381717 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1075/2137] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:52:13 2022] POST /identity/v3/projects => generated 407 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:52:13.386566 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c52b57-d397-45bf-a32d-e52c7ef58bcd admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:13.420626 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:13.456746 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c52b57-d397-45bf-a32d-e52c7ef58bcd admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:13.461536 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41dc28c-dc93-4b2d-a617-acf6ead1ab79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:13.462245 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41dc28c-dc93-4b2d-a617-acf6ead1ab79 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:13.462521 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41dc28c-dc93-4b2d-a617-acf6ead1ab79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:13.462791 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41dc28c-dc93-4b2d-a617-acf6ead1ab79 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:13.463488 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41dc28c-dc93-4b2d-a617-acf6ead1ab79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:13.465453 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41dc28c-dc93-4b2d-a617-acf6ead1ab79 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:13.475386 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1063/2138] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:52:13 2022] POST /identity/v3/users => generated 293 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:52:13.476655 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351b0314-1213-4d51-ac31-3bce2368d60f admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:13.487089 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351b0314-1213-4d51-ac31-3bce2368d60f admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:13.492126 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c44bc01-d24f-4d10-8cdf-bd3c5c28714f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:13.493492 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44bc01-d24f-4d10-8cdf-bd3c5c28714f admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:13.494205 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44bc01-d24f-4d10-8cdf-bd3c5c28714f admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:13.494492 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44bc01-d24f-4d10-8cdf-bd3c5c28714f admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:13.495227 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c44bc01-d24f-4d10-8cdf-bd3c5c28714f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:13.497300 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c44bc01-d24f-4d10-8cdf-bd3c5c28714f admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:13.508017 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1064/2139] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:52:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:52:13.513102 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c44bc01-d24f-4d10-8cdf-bd3c5c28714f admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:13.523714 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c44bc01-d24f-4d10-8cdf-bd3c5c28714f admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:13.528950 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3026a2-34a6-470a-ba87-1a167c8349ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:13.529616 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3026a2-34a6-470a-ba87-1a167c8349ac admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:13.543907 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3026a2-34a6-470a-ba87-1a167c8349ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:13.543907 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3026a2-34a6-470a-ba87-1a167c8349ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:13.543907 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3026a2-34a6-470a-ba87-1a167c8349ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:13.543907 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3026a2-34a6-470a-ba87-1a167c8349ac admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:13.543907 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1065/2140] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:52:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:52:13.547842 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3026a2-34a6-470a-ba87-1a167c8349ac admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:13.559971 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3026a2-34a6-470a-ba87-1a167c8349ac admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:13.566254 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9bd509-d765-446f-96a4-b61c109d2536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:13.567186 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9bd509-d765-446f-96a4-b61c109d2536 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:13.574555 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9bd509-d765-446f-96a4-b61c109d2536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:13.574555 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9bd509-d765-446f-96a4-b61c109d2536 admin admin] PATH_INFO: `/v3/projects/93af4a5c7c424941a435e44272d3f19f/users/34e6033285dd4ecfa61298e562000a49/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:13.575713 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1076/2141] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:52:13 2022] POST /identity/v3/users => generated 310 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:52:13.579581 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41dc28c-dc93-4b2d-a617-acf6ead1ab79 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:13.584343 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9bd509-d765-446f-96a4-b61c109d2536 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93af4a5c7c424941a435e44272d3f19f, user_id=34e6033285dd4ecfa61298e562000a49, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:13.587260 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9bd509-d765-446f-96a4-b61c109d2536 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:13.590177 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41dc28c-dc93-4b2d-a617-acf6ead1ab79 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:13.599807 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44861b9c-abda-4340-bc79-eec6692a097c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:13.599807 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44861b9c-abda-4340-bc79-eec6692a097c admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:13.599807 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44861b9c-abda-4340-bc79-eec6692a097c admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:13.599807 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44861b9c-abda-4340-bc79-eec6692a097c admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:13.600629 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44861b9c-abda-4340-bc79-eec6692a097c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:13.600629 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44861b9c-abda-4340-bc79-eec6692a097c admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:13.613781 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1077/2142] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:52:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:52:13.615160 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44861b9c-abda-4340-bc79-eec6692a097c admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:13.625773 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44861b9c-abda-4340-bc79-eec6692a097c admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:13.632344 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6566e9b3-2792-44eb-83c8-43a471de5b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:13.633211 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6566e9b3-2792-44eb-83c8-43a471de5b09 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:13.633494 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6566e9b3-2792-44eb-83c8-43a471de5b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:13.634620 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6566e9b3-2792-44eb-83c8-43a471de5b09 admin admin] PATH_INFO: `/v3/projects/bb64b47982674c6e8437e89f3fec355c/users/f287d8f2d0ff45ec86c2a0785b87a59b/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:13.650227 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6566e9b3-2792-44eb-83c8-43a471de5b09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb64b47982674c6e8437e89f3fec355c, user_id=f287d8f2d0ff45ec86c2a0785b87a59b, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:13.654916 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6566e9b3-2792-44eb-83c8-43a471de5b09 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:13.699214 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1066/2143] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:52:13 2022] PUT /identity/v3/projects/93af4a5c7c424941a435e44272d3f19f/users/34e6033285dd4ecfa61298e562000a49/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:52:13.709421 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb48333-5690-4534-adfc-408585ea2b89 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:13.709759 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb48333-5690-4534-adfc-408585ea2b89 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:13.710059 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb48333-5690-4534-adfc-408585ea2b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:13.737762 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-4bb48333-5690-4534-adfc-408585ea2b89 None None] MFA Rules not processed for user `34e6033285dd4ecfa61298e562000a49`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:13.739052 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-4bb48333-5690-4534-adfc-408585ea2b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:13.777169 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1078/2144] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:52:13 2022] PUT /identity/v3/projects/bb64b47982674c6e8437e89f3fec355c/users/f287d8f2d0ff45ec86c2a0785b87a59b/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:13.779017 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6566e9b3-2792-44eb-83c8-43a471de5b09 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:13.811128 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1067/2145] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:52:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:52:13.820532 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb48333-5690-4534-adfc-408585ea2b89 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:13.865275 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6566e9b3-2792-44eb-83c8-43a471de5b09 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:13.870909 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab696d2-f41b-481b-ab31-99ba0acab471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:13.871666 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab696d2-f41b-481b-ab31-99ba0acab471 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:13.871994 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab696d2-f41b-481b-ab31-99ba0acab471 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:13.872356 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab696d2-f41b-481b-ab31-99ba0acab471 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:13.873412 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab696d2-f41b-481b-ab31-99ba0acab471 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:13.875040 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab696d2-f41b-481b-ab31-99ba0acab471 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:13.888274 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1079/2146] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:52:13 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:52:13.892421 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab696d2-f41b-481b-ab31-99ba0acab471 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:13.902562 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb48333-5690-4534-adfc-408585ea2b89 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:13.904797 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab696d2-f41b-481b-ab31-99ba0acab471 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:13.907606 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15387ea-a657-41e0-a429-df3d980371b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:13.908386 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15387ea-a657-41e0-a429-df3d980371b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:13.908754 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15387ea-a657-41e0-a429-df3d980371b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:13.909175 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15387ea-a657-41e0-a429-df3d980371b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:13.909952 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15387ea-a657-41e0-a429-df3d980371b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:13.910062 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a535f662-f1f3-47ba-8e58-4e759ade3eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:13.911151 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a535f662-f1f3-47ba-8e58-4e759ade3eb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:13.911430 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a535f662-f1f3-47ba-8e58-4e759ade3eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:13.911708 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a535f662-f1f3-47ba-8e58-4e759ade3eb9 admin admin] PATH_INFO: `/v3/projects/bb64b47982674c6e8437e89f3fec355c/users/f287d8f2d0ff45ec86c2a0785b87a59b/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:13.912306 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15387ea-a657-41e0-a429-df3d980371b1 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:13.913870 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a535f662-f1f3-47ba-8e58-4e759ade3eb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb64b47982674c6e8437e89f3fec355c, user_id=f287d8f2d0ff45ec86c2a0785b87a59b, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:13.917575 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a535f662-f1f3-47ba-8e58-4e759ade3eb9 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:13.930053 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1080/2147] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:52:13 2022] PUT /identity/v3/projects/bb64b47982674c6e8437e89f3fec355c/users/f287d8f2d0ff45ec86c2a0785b87a59b/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:52:13.941209 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58c817-5bfe-4f03-87a4-6e642435e2ce None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:13.941501 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58c817-5bfe-4f03-87a4-6e642435e2ce None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:13.941866 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58c817-5bfe-4f03-87a4-6e642435e2ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:13.970817 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-aa58c817-5bfe-4f03-87a4-6e642435e2ce None None] MFA Rules not processed for user `f287d8f2d0ff45ec86c2a0785b87a59b`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:13.972034 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-aa58c817-5bfe-4f03-87a4-6e642435e2ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:14.025844 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:14.026989 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:14.036503 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1068/2148] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:52:13 2022] POST /identity/v3/projects => generated 387 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:52:14.040878 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15387ea-a657-41e0-a429-df3d980371b1 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:14.061147 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1081/2149] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:52:13 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:52:14.075600 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa58c817-5bfe-4f03-87a4-6e642435e2ce None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:14.077899 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-aa58c817-5bfe-4f03-87a4-6e642435e2ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:14.118990 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15387ea-a657-41e0-a429-df3d980371b1 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:14.125458 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e591d75-d307-4e8c-ac52-be4e2c999b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:14.126146 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e591d75-d307-4e8c-ac52-be4e2c999b5c admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:14.126429 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e591d75-d307-4e8c-ac52-be4e2c999b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:14.126778 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e591d75-d307-4e8c-ac52-be4e2c999b5c admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:14.127417 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e591d75-d307-4e8c-ac52-be4e2c999b5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:14.129545 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e591d75-d307-4e8c-ac52-be4e2c999b5c admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:14.178432 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa58c817-5bfe-4f03-87a4-6e642435e2ce None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:14.232917 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1069/2150] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:52:14 2022] POST /identity/v3/users => generated 294 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:52:14.236782 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e591d75-d307-4e8c-ac52-be4e2c999b5c admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:14.236958 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d26366b-fd91-4d00-8906-61e8fe89b2b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:14.237457 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26366b-fd91-4d00-8906-61e8fe89b2b6 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:14.237744 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26366b-fd91-4d00-8906-61e8fe89b2b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:14.238027 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26366b-fd91-4d00-8906-61e8fe89b2b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:14.247281 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e591d75-d307-4e8c-ac52-be4e2c999b5c admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:14.253341 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d957da-2dbf-497e-bd17-c9c63d1ac602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:14.258240 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d957da-2dbf-497e-bd17-c9c63d1ac602 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:14.258240 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d957da-2dbf-497e-bd17-c9c63d1ac602 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:14.258240 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d957da-2dbf-497e-bd17-c9c63d1ac602 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:14.258240 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d957da-2dbf-497e-bd17-c9c63d1ac602 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:14.258240 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d957da-2dbf-497e-bd17-c9c63d1ac602 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:14.268409 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1070/2151] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:52:14.272465 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d957da-2dbf-497e-bd17-c9c63d1ac602 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:14.283438 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d957da-2dbf-497e-bd17-c9c63d1ac602 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:14.288448 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b99e4c4-1ecb-446b-bd46-39e4ff6f2f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:14.289116 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b99e4c4-1ecb-446b-bd46-39e4ff6f2f13 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:14.289409 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b99e4c4-1ecb-446b-bd46-39e4ff6f2f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:14.289690 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b99e4c4-1ecb-446b-bd46-39e4ff6f2f13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:14.291147 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b99e4c4-1ecb-446b-bd46-39e4ff6f2f13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:14.292920 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b99e4c4-1ecb-446b-bd46-39e4ff6f2f13 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:14.304115 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1071/2152] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:52:14.304727 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d26366b-fd91-4d00-8906-61e8fe89b2b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:14.306824 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d26366b-fd91-4d00-8906-61e8fe89b2b6 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:14.308079 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b99e4c4-1ecb-446b-bd46-39e4ff6f2f13 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:14.318203 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b99e4c4-1ecb-446b-bd46-39e4ff6f2f13 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:14.318435 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1082/2153] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:52:14 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:52:14.323101 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ac5b4c-72c1-4abc-bb7f-5047da2d0cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:14.323999 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ac5b4c-72c1-4abc-bb7f-5047da2d0cc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:14.324409 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ac5b4c-72c1-4abc-bb7f-5047da2d0cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:14.324572 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ac5b4c-72c1-4abc-bb7f-5047da2d0cc7 admin admin] PATH_INFO: `/v3/projects/757549c6346a41ac8960423009ad3524/users/81dd0abdb9b14ee880f1c652f53e737d/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:14.342227 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ac5b4c-72c1-4abc-bb7f-5047da2d0cc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=757549c6346a41ac8960423009ad3524, user_id=81dd0abdb9b14ee880f1c652f53e737d, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:14.344922 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ac5b4c-72c1-4abc-bb7f-5047da2d0cc7 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:14.440115 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1072/2154] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:52:14 2022] PUT /identity/v3/projects/757549c6346a41ac8960423009ad3524/users/81dd0abdb9b14ee880f1c652f53e737d/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:52:14.451108 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2980b93d-882e-46fb-9a64-8b3fa5ecd21d None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:14.451394 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2980b93d-882e-46fb-9a64-8b3fa5ecd21d None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:14.451671 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2980b93d-882e-46fb-9a64-8b3fa5ecd21d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:14.481015 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-2980b93d-882e-46fb-9a64-8b3fa5ecd21d None None] MFA Rules not processed for user `81dd0abdb9b14ee880f1c652f53e737d`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:14.482269 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-2980b93d-882e-46fb-9a64-8b3fa5ecd21d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:14.533724 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:14.555577 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1083/2155] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:52:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:14.560201 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ac5b4c-72c1-4abc-bb7f-5047da2d0cc7 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:14.630308 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ac5b4c-72c1-4abc-bb7f-5047da2d0cc7 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:14.637616 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4823bb-d828-4a9a-97d1-7ec6a7614b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:14.638295 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4823bb-d828-4a9a-97d1-7ec6a7614b1c admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:14.638578 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4823bb-d828-4a9a-97d1-7ec6a7614b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:14.638853 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4823bb-d828-4a9a-97d1-7ec6a7614b1c admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:14.639600 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4823bb-d828-4a9a-97d1-7ec6a7614b1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:14.642312 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4823bb-d828-4a9a-97d1-7ec6a7614b1c admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:14.752451 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2980b93d-882e-46fb-9a64-8b3fa5ecd21d None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:14.786815 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1073/2156] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:52:14 2022] POST /identity/v3/projects => generated 385 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:52:14.788183 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4823bb-d828-4a9a-97d1-7ec6a7614b1c admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:14.819230 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2980b93d-882e-46fb-9a64-8b3fa5ecd21d None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:14.820797 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:14.857039 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4823bb-d828-4a9a-97d1-7ec6a7614b1c admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:14.861801 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c60401-b60a-4376-9289-09f85af51641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:14.862544 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c60401-b60a-4376-9289-09f85af51641 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:14.862790 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c60401-b60a-4376-9289-09f85af51641 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:14.863075 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c60401-b60a-4376-9289-09f85af51641 admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:14.863761 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c60401-b60a-4376-9289-09f85af51641 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:14.865720 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c60401-b60a-4376-9289-09f85af51641 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:14.872985 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef943985-072f-4f9c-b1d5-979779c9101a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:14.873685 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef943985-072f-4f9c-b1d5-979779c9101a service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:14.873968 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef943985-072f-4f9c-b1d5-979779c9101a service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:14.874248 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef943985-072f-4f9c-b1d5-979779c9101a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:14.937977 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef943985-072f-4f9c-b1d5-979779c9101a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:14.939468 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef943985-072f-4f9c-b1d5-979779c9101a service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:14.950455 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1084/2157] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:52:14 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:52:15.000004 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1074/2158] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:52:14 2022] POST /identity/v3/users => generated 299 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:52:15.003895 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef943985-072f-4f9c-b1d5-979779c9101a service cinder] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:15.019188 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef943985-072f-4f9c-b1d5-979779c9101a service cinder] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:15.024315 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1103861f-464f-4a57-a561-62098d7e5fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:15.024903 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1103861f-464f-4a57-a561-62098d7e5fb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:15.025189 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1103861f-464f-4a57-a561-62098d7e5fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:15.025469 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1103861f-464f-4a57-a561-62098d7e5fb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:15.026210 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1103861f-464f-4a57-a561-62098d7e5fb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:15.027839 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1103861f-464f-4a57-a561-62098d7e5fb1 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:15.043643 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1085/2159] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:52:15 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:52:15.048354 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c60401-b60a-4376-9289-09f85af51641 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:15.066124 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c60401-b60a-4376-9289-09f85af51641 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:15.075153 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1051d8-d9e6-4e7c-94dc-7891b2d83200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:15.076266 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1051d8-d9e6-4e7c-94dc-7891b2d83200 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:15.078824 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1051d8-d9e6-4e7c-94dc-7891b2d83200 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:15.078824 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1051d8-d9e6-4e7c-94dc-7891b2d83200 admin admin] PATH_INFO: `/v3/projects/02115930a70c435f88324970dcc3c3e9/users/6a5affc371574927b149409da8df2670/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:15.104244 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1051d8-d9e6-4e7c-94dc-7891b2d83200 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02115930a70c435f88324970dcc3c3e9, user_id=6a5affc371574927b149409da8df2670, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:15.108244 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1051d8-d9e6-4e7c-94dc-7891b2d83200 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:15.197630 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1075/2160] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:52:15 2022] PUT /identity/v3/projects/02115930a70c435f88324970dcc3c3e9/users/6a5affc371574927b149409da8df2670/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:15.201557 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1103861f-464f-4a57-a561-62098d7e5fb1 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:15.212942 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1103861f-464f-4a57-a561-62098d7e5fb1 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:15.259435 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:15.275202 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4411045-f3ec-4d43-82e3-52370929b130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:15.275873 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4411045-f3ec-4d43-82e3-52370929b130 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:15.276183 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4411045-f3ec-4d43-82e3-52370929b130 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:15.276459 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4411045-f3ec-4d43-82e3-52370929b130 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:15.277183 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4411045-f3ec-4d43-82e3-52370929b130 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:15.278776 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4411045-f3ec-4d43-82e3-52370929b130 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:15.288692 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1086/2161] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:52:15 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:52:15.293503 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1051d8-d9e6-4e7c-94dc-7891b2d83200 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:15.303906 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1051d8-d9e6-4e7c-94dc-7891b2d83200 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:15.309637 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a943b9a-c3f3-4175-9625-30ded6f463d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:15.310600 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a943b9a-c3f3-4175-9625-30ded6f463d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:15.310892 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a943b9a-c3f3-4175-9625-30ded6f463d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:15.311168 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a943b9a-c3f3-4175-9625-30ded6f463d4 admin admin] PATH_INFO: `/v3/projects/02115930a70c435f88324970dcc3c3e9/users/6a5affc371574927b149409da8df2670/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:15.313209 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a943b9a-c3f3-4175-9625-30ded6f463d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02115930a70c435f88324970dcc3c3e9, user_id=6a5affc371574927b149409da8df2670, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:15.315973 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a943b9a-c3f3-4175-9625-30ded6f463d4 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:15.326974 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1076/2162] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:52:15 2022] PUT /identity/v3/projects/02115930a70c435f88324970dcc3c3e9/users/6a5affc371574927b149409da8df2670/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:15.338750 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadab07b-f5b3-4af9-82be-5735fb3f231d None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:15.339045 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadab07b-f5b3-4af9-82be-5735fb3f231d None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:15.339323 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadab07b-f5b3-4af9-82be-5735fb3f231d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:15.366898 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-eadab07b-f5b3-4af9-82be-5735fb3f231d None None] MFA Rules not processed for user `6a5affc371574927b149409da8df2670`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:15.369310 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-eadab07b-f5b3-4af9-82be-5735fb3f231d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:15.439991 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1087/2163] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:52:15 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:15.450746 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a943b9a-c3f3-4175-9625-30ded6f463d4 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:15.452471 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-1a943b9a-c3f3-4175-9625-30ded6f463d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:15.510005 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a943b9a-c3f3-4175-9625-30ded6f463d4 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:15.514019 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5675d6-73e8-49c7-8399-ab2f86914af1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:15.514664 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5675d6-73e8-49c7-8399-ab2f86914af1 service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:15.514946 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5675d6-73e8-49c7-8399-ab2f86914af1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:15.515244 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5675d6-73e8-49c7-8399-ab2f86914af1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:15.524992 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5675d6-73e8-49c7-8399-ab2f86914af1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:15.526417 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5675d6-73e8-49c7-8399-ab2f86914af1 service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:15.536150 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1077/2164] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:52:15 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:52:16.715030 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadab07b-f5b3-4af9-82be-5735fb3f231d None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:16.724844 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadab07b-f5b3-4af9-82be-5735fb3f231d None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:16.729108 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be53794-09f3-48cd-bf35-0b62369fd1f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:16.729772 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be53794-09f3-48cd-bf35-0b62369fd1f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:16.730055 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be53794-09f3-48cd-bf35-0b62369fd1f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:16.730333 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be53794-09f3-48cd-bf35-0b62369fd1f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:16.799122 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be53794-09f3-48cd-bf35-0b62369fd1f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:16.800612 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be53794-09f3-48cd-bf35-0b62369fd1f6 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:16.810995 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1088/2165] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:52:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:52:17.624697 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5675d6-73e8-49c7-8399-ab2f86914af1 service cinder] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:17.626173 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-1d5675d6-73e8-49c7-8399-ab2f86914af1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:17.705068 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5675d6-73e8-49c7-8399-ab2f86914af1 service cinder] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:17.710035 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37b1a7b-0a17-4fc4-91b0-bfb7261f033c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:17.710899 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b1a7b-0a17-4fc4-91b0-bfb7261f033c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:17.711181 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b1a7b-0a17-4fc4-91b0-bfb7261f033c admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:17.711463 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b1a7b-0a17-4fc4-91b0-bfb7261f033c admin admin] PATH_INFO: `/v3/users/e95d993b6ed24073bbf1618ab4d1118e` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:17.714060 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37b1a7b-0a17-4fc4-91b0-bfb7261f033c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e95d993b6ed24073bbf1618ab4d1118e)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:17.715985 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37b1a7b-0a17-4fc4-91b0-bfb7261f033c admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:17.742137 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:17.925110 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-e37b1a7b-0a17-4fc4-91b0-bfb7261f033c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e95d993b6ed24073bbf1618ab4d1118e'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:17.925835 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-e37b1a7b-0a17-4fc4-91b0-bfb7261f033c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e95d993b6ed24073bbf1618ab4d1118e'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:17.935693 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-e37b1a7b-0a17-4fc4-91b0-bfb7261f033c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e95d993b6ed24073bbf1618ab4d1118e'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:17.936422 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-e37b1a7b-0a17-4fc4-91b0-bfb7261f033c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e95d993b6ed24073bbf1618ab4d1118e'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:18.048498 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-e37b1a7b-0a17-4fc4-91b0-bfb7261f033c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e95d993b6ed24073bbf1618ab4d1118e'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:18.069536 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1078/2166] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:52:17 2022] DELETE /identity/v3/users/e95d993b6ed24073bbf1618ab4d1118e => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:18.370948 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be53794-09f3-48cd-bf35-0b62369fd1f6 service cinder] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:18.384018 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be53794-09f3-48cd-bf35-0b62369fd1f6 service cinder] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:18.451661 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742fc315-3c99-427a-aaac-4c4df2e05ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:18.452717 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742fc315-3c99-427a-aaac-4c4df2e05ed1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:18.452996 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742fc315-3c99-427a-aaac-4c4df2e05ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:18.453269 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742fc315-3c99-427a-aaac-4c4df2e05ed1 admin admin] PATH_INFO: `/v3/projects/0e498ac8b52d4a068016e44c094996c1` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:18.454456 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742fc315-3c99-427a-aaac-4c4df2e05ed1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e498ac8b52d4a068016e44c094996c1)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:18.456433 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742fc315-3c99-427a-aaac-4c4df2e05ed1 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:18.464003 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:18.598265 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-742fc315-3c99-427a-aaac-4c4df2e05ed1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e498ac8b52d4a068016e44c094996c1', 'request_id': 'req-742fc315-3c99-427a-aaac-4c4df2e05ed1'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:18.606970 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-742fc315-3c99-427a-aaac-4c4df2e05ed1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e498ac8b52d4a068016e44c094996c1', 'request_id': 'req-742fc315-3c99-427a-aaac-4c4df2e05ed1'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:18.710185 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-742fc315-3c99-427a-aaac-4c4df2e05ed1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e498ac8b52d4a068016e44c094996c1', 'request_id': 'req-742fc315-3c99-427a-aaac-4c4df2e05ed1'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:18.723262 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-742fc315-3c99-427a-aaac-4c4df2e05ed1 admin admin] The token cache is being invalidate because project 0e498ac8b52d4a068016e44c094996c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:52:18.723557 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-742fc315-3c99-427a-aaac-4c4df2e05ed1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:18.724233 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-742fc315-3c99-427a-aaac-4c4df2e05ed1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:18.725817 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1089/2167] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:52:18 2022] DELETE /identity/v3/projects/0e498ac8b52d4a068016e44c094996c1 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:52:18.730676 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793622c2-6fea-4cf2-9ec8-fb06aaa56a18 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:18.731152 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793622c2-6fea-4cf2-9ec8-fb06aaa56a18 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:18.731390 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793622c2-6fea-4cf2-9ec8-fb06aaa56a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:18.749611 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-793622c2-6fea-4cf2-9ec8-fb06aaa56a18 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:18.750828 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-793622c2-6fea-4cf2-9ec8-fb06aaa56a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:18.813371 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1079/2168] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:52:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:18.824163 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae2dd8d-f1b0-43a2-9923-ec8f7bc2bf72 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:18.824721 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae2dd8d-f1b0-43a2-9923-ec8f7bc2bf72 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:18.825280 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae2dd8d-f1b0-43a2-9923-ec8f7bc2bf72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:18.853346 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-fae2dd8d-f1b0-43a2-9923-ec8f7bc2bf72 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:18.854586 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-fae2dd8d-f1b0-43a2-9923-ec8f7bc2bf72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:18.863196 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1090/2169] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:52:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:18.867661 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793622c2-6fea-4cf2-9ec8-fb06aaa56a18 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:18.878810 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793622c2-6fea-4cf2-9ec8-fb06aaa56a18 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:18.885319 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435b7c33-bcb7-459a-9289-91971ff0dbdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:18.886355 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435b7c33-bcb7-459a-9289-91971ff0dbdf admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:18.886638 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435b7c33-bcb7-459a-9289-91971ff0dbdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:18.886925 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435b7c33-bcb7-459a-9289-91971ff0dbdf admin admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:18.887753 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435b7c33-bcb7-459a-9289-91971ff0dbdf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:52:18.888154 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435b7c33-bcb7-459a-9289-91971ff0dbdf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:18.889831 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435b7c33-bcb7-459a-9289-91971ff0dbdf admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:18.906764 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1080/2170] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jun 15 14:52:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:52:18.911196 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae2dd8d-f1b0-43a2-9923-ec8f7bc2bf72 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:18.921930 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae2dd8d-f1b0-43a2-9923-ec8f7bc2bf72 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:18.927356 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7a6cbd-4c55-44de-bdca-a10c98c76779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:18.928029 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a6cbd-4c55-44de-bdca-a10c98c76779 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:18.928344 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a6cbd-4c55-44de-bdca-a10c98c76779 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:18.928762 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a6cbd-4c55-44de-bdca-a10c98c76779 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:18.929478 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7a6cbd-4c55-44de-bdca-a10c98c76779 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:18.931980 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7a6cbd-4c55-44de-bdca-a10c98c76779 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:19.022453 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1091/2171] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:52:18 2022] POST /identity/v3/projects => generated 389 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:52:19.027470 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435b7c33-bcb7-459a-9289-91971ff0dbdf admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:19.038075 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435b7c33-bcb7-459a-9289-91971ff0dbdf admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:19.102662 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c11a14-2c02-4801-9e9f-cf1bc7b8ce67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:19.103338 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c11a14-2c02-4801-9e9f-cf1bc7b8ce67 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:19.103622 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c11a14-2c02-4801-9e9f-cf1bc7b8ce67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:19.103903 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c11a14-2c02-4801-9e9f-cf1bc7b8ce67 admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:19.104631 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c11a14-2c02-4801-9e9f-cf1bc7b8ce67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:19.106574 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c11a14-2c02-4801-9e9f-cf1bc7b8ce67 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:19.214226 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1081/2172] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:52:19 2022] POST /identity/v3/users => generated 295 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:52:19.218397 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7a6cbd-4c55-44de-bdca-a10c98c76779 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:19.229269 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7a6cbd-4c55-44de-bdca-a10c98c76779 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:19.234050 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e55699-7454-4d6d-a448-80ec79764c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:19.234718 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e55699-7454-4d6d-a448-80ec79764c40 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:19.235078 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e55699-7454-4d6d-a448-80ec79764c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:19.235266 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e55699-7454-4d6d-a448-80ec79764c40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:19.235987 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e55699-7454-4d6d-a448-80ec79764c40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:19.237623 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e55699-7454-4d6d-a448-80ec79764c40 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:19.250837 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1092/2173] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:52:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:52:19.255095 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c11a14-2c02-4801-9e9f-cf1bc7b8ce67 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:19.267636 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c11a14-2c02-4801-9e9f-cf1bc7b8ce67 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:19.272854 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578fef04-a409-439e-8418-26f5b5254b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:19.273313 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578fef04-a409-439e-8418-26f5b5254b95 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:19.273592 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578fef04-a409-439e-8418-26f5b5254b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:19.273874 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578fef04-a409-439e-8418-26f5b5254b95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:19.274894 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578fef04-a409-439e-8418-26f5b5254b95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:19.276659 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578fef04-a409-439e-8418-26f5b5254b95 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:19.289884 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1082/2174] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:52:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:52:19.294245 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e55699-7454-4d6d-a448-80ec79764c40 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:19.304698 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e55699-7454-4d6d-a448-80ec79764c40 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:19.310254 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba61178c-1a9d-4b7b-9c74-89acb1645bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:19.311150 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba61178c-1a9d-4b7b-9c74-89acb1645bd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:19.311475 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba61178c-1a9d-4b7b-9c74-89acb1645bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:19.311780 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba61178c-1a9d-4b7b-9c74-89acb1645bd8 admin admin] PATH_INFO: `/v3/projects/f9c28220b6ea4db99f81a37e0794450b/users/2dcc02eb960e45b99587438aa3696f40/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:19.327608 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba61178c-1a9d-4b7b-9c74-89acb1645bd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9c28220b6ea4db99f81a37e0794450b, user_id=2dcc02eb960e45b99587438aa3696f40, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:19.330368 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba61178c-1a9d-4b7b-9c74-89acb1645bd8 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:19.442082 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1093/2175] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:52:19 2022] PUT /identity/v3/projects/f9c28220b6ea4db99f81a37e0794450b/users/2dcc02eb960e45b99587438aa3696f40/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:19.448552 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea288e-031b-4211-8635-d77caa0d67d9 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:19.448842 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea288e-031b-4211-8635-d77caa0d67d9 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:19.449123 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea288e-031b-4211-8635-d77caa0d67d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:19.461697 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:19.467257 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-d1ea288e-031b-4211-8635-d77caa0d67d9 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:19.468497 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-d1ea288e-031b-4211-8635-d77caa0d67d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:19.528257 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1083/2176] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:52:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:19.538123 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba61178c-1a9d-4b7b-9c74-89acb1645bd8 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:19.540268 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-ba61178c-1a9d-4b7b-9c74-89acb1645bd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:19.609941 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba61178c-1a9d-4b7b-9c74-89acb1645bd8 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:19.616717 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5aa1c5-8ca3-4858-a49d-f7c8a8cd595f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:19.617752 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5aa1c5-8ca3-4858-a49d-f7c8a8cd595f service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:19.617752 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5aa1c5-8ca3-4858-a49d-f7c8a8cd595f service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:19.618511 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5aa1c5-8ca3-4858-a49d-f7c8a8cd595f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:19.633033 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5aa1c5-8ca3-4858-a49d-f7c8a8cd595f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:19.635136 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5aa1c5-8ca3-4858-a49d-f7c8a8cd595f service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:19.646123 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1094/2177] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:52:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:52:25.410694 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f537fa35-975b-4763-ab01-185c032964d9 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:25.412824 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f537fa35-975b-4763-ab01-185c032964d9 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:25.413308 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f537fa35-975b-4763-ab01-185c032964d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:25.452127 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-f537fa35-975b-4763-ab01-185c032964d9 None None] MFA Rules not processed for user `2dcc02eb960e45b99587438aa3696f40`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:25.453511 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-f537fa35-975b-4763-ab01-185c032964d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:25.554411 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1084/2178] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:52:25 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:25.563212 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5aa1c5-8ca3-4858-a49d-f7c8a8cd595f service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:25.571709 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-cb5aa1c5-8ca3-4858-a49d-f7c8a8cd595f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:25.642287 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:25.738709 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5aa1c5-8ca3-4858-a49d-f7c8a8cd595f service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:25.745220 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fbed42-a1d0-4b8d-874c-63e603c8b4d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0927c05c340c48e48b18754888aa9c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:25.745885 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fbed42-a1d0-4b8d-874c-63e603c8b4d5 service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:25.746179 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fbed42-a1d0-4b8d-874c-63e603c8b4d5 service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:25.746440 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fbed42-a1d0-4b8d-874c-63e603c8b4d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:25.755902 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fbed42-a1d0-4b8d-874c-63e603c8b4d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:25.762045 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fbed42-a1d0-4b8d-874c-63e603c8b4d5 service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:25.769687 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1095/2179] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:52:25 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:52:28.579397 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a795b-5324-439a-bfec-b70eb1070323 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:28.580193 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a795b-5324-439a-bfec-b70eb1070323 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:28.580193 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a795b-5324-439a-bfec-b70eb1070323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:28.601758 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-457a795b-5324-439a-bfec-b70eb1070323 None None] MFA Rules not processed for user `2dcc02eb960e45b99587438aa3696f40`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:28.603281 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-457a795b-5324-439a-bfec-b70eb1070323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:28.610418 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1085/2180] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:52:28 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:52:29.378311 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fbed42-a1d0-4b8d-874c-63e603c8b4d5 service nova] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:29.380974 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-96fbed42-a1d0-4b8d-874c-63e603c8b4d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:29.405283 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fbed42-a1d0-4b8d-874c-63e603c8b4d5 service nova] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:29.410933 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d17c286-4503-43cb-804e-9fa948af6c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:29.411983 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d17c286-4503-43cb-804e-9fa948af6c10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:29.412484 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d17c286-4503-43cb-804e-9fa948af6c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:29.413031 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d17c286-4503-43cb-804e-9fa948af6c10 admin admin] PATH_INFO: `/v3/users/b165fd1ff9f94d42b139672a84f770b2` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:29.417196 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d17c286-4503-43cb-804e-9fa948af6c10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b165fd1ff9f94d42b139672a84f770b2)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:29.419461 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d17c286-4503-43cb-804e-9fa948af6c10 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:29.723441 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457a795b-5324-439a-bfec-b70eb1070323 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:29.728197 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-457a795b-5324-439a-bfec-b70eb1070323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:29.756966 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-2d17c286-4503-43cb-804e-9fa948af6c10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b165fd1ff9f94d42b139672a84f770b2'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:29.766597 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-2d17c286-4503-43cb-804e-9fa948af6c10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b165fd1ff9f94d42b139672a84f770b2'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:29.768369 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-2d17c286-4503-43cb-804e-9fa948af6c10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b165fd1ff9f94d42b139672a84f770b2'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:29.860745 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-2d17c286-4503-43cb-804e-9fa948af6c10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b165fd1ff9f94d42b139672a84f770b2'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:29.892355 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457a795b-5324-439a-bfec-b70eb1070323 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:29.904061 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-2d17c286-4503-43cb-804e-9fa948af6c10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b165fd1ff9f94d42b139672a84f770b2'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:29.904360 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f87ac7d-781b-4b75-86ea-9da21c18f8e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:29.905205 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f87ac7d-781b-4b75-86ea-9da21c18f8e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:29.905491 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f87ac7d-781b-4b75-86ea-9da21c18f8e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:29.905768 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f87ac7d-781b-4b75-86ea-9da21c18f8e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:29.914585 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-4f87ac7d-781b-4b75-86ea-9da21c18f8e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:29.922798 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1096/2181] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:52:29 2022] DELETE /identity/v3/users/b165fd1ff9f94d42b139672a84f770b2 => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:29.932843 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b86475-b566-4efc-aed1-de50fe22dadc None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:29.934987 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b86475-b566-4efc-aed1-de50fe22dadc None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:29.936761 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b86475-b566-4efc-aed1-de50fe22dadc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:29.966443 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-68b86475-b566-4efc-aed1-de50fe22dadc None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:29.969776 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-68b86475-b566-4efc-aed1-de50fe22dadc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:30.044500 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:30.064362 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f87ac7d-781b-4b75-86ea-9da21c18f8e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:30.069185 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f87ac7d-781b-4b75-86ea-9da21c18f8e6 service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:30.088534 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1086/2182] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:52:29 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:52:30.130867 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f87ac7d-781b-4b75-86ea-9da21c18f8e6 service cinder] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:30.134066 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1097/2183] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:52:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:30.134944 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-4f87ac7d-781b-4b75-86ea-9da21c18f8e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:30.156429 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f87ac7d-781b-4b75-86ea-9da21c18f8e6 service cinder] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:30.244050 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a86a9a-1b35-41dd-b755-8af1eaa22489 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:30.245145 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a86a9a-1b35-41dd-b755-8af1eaa22489 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:30.245669 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a86a9a-1b35-41dd-b755-8af1eaa22489 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:30.246029 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a86a9a-1b35-41dd-b755-8af1eaa22489 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:30.263821 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a86a9a-1b35-41dd-b755-8af1eaa22489 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:30.271179 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a86a9a-1b35-41dd-b755-8af1eaa22489 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:30.287321 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1087/2184] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:52:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:52:30.887502 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b86475-b566-4efc-aed1-de50fe22dadc None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:30.889670 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-68b86475-b566-4efc-aed1-de50fe22dadc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:30.901704 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:30.908077 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b86475-b566-4efc-aed1-de50fe22dadc None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:30.914182 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a444a0-fa61-4377-bdb0-8b3904b48fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:30.916825 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a444a0-fa61-4377-bdb0-8b3904b48fc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:30.917313 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a444a0-fa61-4377-bdb0-8b3904b48fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:30.917764 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a444a0-fa61-4377-bdb0-8b3904b48fc5 admin admin] PATH_INFO: `/v3/projects/aa4bcabaca5641f3b5372df5332d948a` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:30.919569 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a444a0-fa61-4377-bdb0-8b3904b48fc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa4bcabaca5641f3b5372df5332d948a)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:30.929506 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a444a0-fa61-4377-bdb0-8b3904b48fc5 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:31.090146 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-f9a444a0-fa61-4377-bdb0-8b3904b48fc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa4bcabaca5641f3b5372df5332d948a', 'request_id': 'req-f9a444a0-fa61-4377-bdb0-8b3904b48fc5'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:31.096351 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-f9a444a0-fa61-4377-bdb0-8b3904b48fc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa4bcabaca5641f3b5372df5332d948a', 'request_id': 'req-f9a444a0-fa61-4377-bdb0-8b3904b48fc5'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:31.162828 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-f9a444a0-fa61-4377-bdb0-8b3904b48fc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa4bcabaca5641f3b5372df5332d948a', 'request_id': 'req-f9a444a0-fa61-4377-bdb0-8b3904b48fc5'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:31.178456 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-f9a444a0-fa61-4377-bdb0-8b3904b48fc5 admin admin] The token cache is being invalidate because project aa4bcabaca5641f3b5372df5332d948a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:52:31.178633 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-f9a444a0-fa61-4377-bdb0-8b3904b48fc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:31.179582 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-f9a444a0-fa61-4377-bdb0-8b3904b48fc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:31.181860 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1098/2185] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:52:30 2022] DELETE /identity/v3/projects/aa4bcabaca5641f3b5372df5332d948a => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:31.978788 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a86a9a-1b35-41dd-b755-8af1eaa22489 service neutron] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:31.980248 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-32a86a9a-1b35-41dd-b755-8af1eaa22489 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:31.996397 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a86a9a-1b35-41dd-b755-8af1eaa22489 service neutron] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:32.054271 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7cbbf6-a1d0-4d10-938b-22753152abbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:32.055677 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7cbbf6-a1d0-4d10-938b-22753152abbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:32.055964 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7cbbf6-a1d0-4d10-938b-22753152abbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:32.056273 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7cbbf6-a1d0-4d10-938b-22753152abbd admin admin] PATH_INFO: `/v3/users/6cdea625d3a74f2babd149314df03cb7` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:32.057641 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7cbbf6-a1d0-4d10-938b-22753152abbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=6cdea625d3a74f2babd149314df03cb7)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:32.059658 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7cbbf6-a1d0-4d10-938b-22753152abbd admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:32.343111 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-ca7cbbf6-a1d0-4d10-938b-22753152abbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6cdea625d3a74f2babd149314df03cb7'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:32.343783 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-ca7cbbf6-a1d0-4d10-938b-22753152abbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6cdea625d3a74f2babd149314df03cb7'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:32.351508 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-ca7cbbf6-a1d0-4d10-938b-22753152abbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6cdea625d3a74f2babd149314df03cb7'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:32.352296 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-ca7cbbf6-a1d0-4d10-938b-22753152abbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6cdea625d3a74f2babd149314df03cb7'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:32.518432 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-ca7cbbf6-a1d0-4d10-938b-22753152abbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6cdea625d3a74f2babd149314df03cb7'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:32.557584 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1088/2186] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:52:31 2022] DELETE /identity/v3/users/6cdea625d3a74f2babd149314df03cb7 => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:32.563139 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6980faf-55b8-4c59-b3ea-dd0c7489af87 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:32.563676 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6980faf-55b8-4c59-b3ea-dd0c7489af87 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:32.564222 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6980faf-55b8-4c59-b3ea-dd0c7489af87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:32.586425 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-f6980faf-55b8-4c59-b3ea-dd0c7489af87 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:32.588655 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-f6980faf-55b8-4c59-b3ea-dd0c7489af87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:32.675390 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1099/2187] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:52:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:32.686219 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7cbbf6-a1d0-4d10-938b-22753152abbd admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:32.687671 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-ca7cbbf6-a1d0-4d10-938b-22753152abbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:32.698328 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:32.745483 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7cbbf6-a1d0-4d10-938b-22753152abbd admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:32.749109 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70aaac2-6f34-484f-9f3d-406b8c29bc5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:32.749755 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70aaac2-6f34-484f-9f3d-406b8c29bc5b service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:32.750039 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70aaac2-6f34-484f-9f3d-406b8c29bc5b service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:32.750317 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70aaac2-6f34-484f-9f3d-406b8c29bc5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:32.759797 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70aaac2-6f34-484f-9f3d-406b8c29bc5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:32.765982 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70aaac2-6f34-484f-9f3d-406b8c29bc5b service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:32.771068 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1089/2188] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:52:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:52:33.220916 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6980faf-55b8-4c59-b3ea-dd0c7489af87 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:33.227168 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-f6980faf-55b8-4c59-b3ea-dd0c7489af87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:33.248571 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6980faf-55b8-4c59-b3ea-dd0c7489af87 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:33.259883 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c5567d-0192-44c4-b157-926a2473d0c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:33.261658 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c5567d-0192-44c4-b157-926a2473d0c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:33.261658 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c5567d-0192-44c4-b157-926a2473d0c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:33.265230 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c5567d-0192-44c4-b157-926a2473d0c0 admin admin] PATH_INFO: `/v3/projects/a105d0e9d4074f2ba1c131df2a5a9fe9` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:33.265230 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c5567d-0192-44c4-b157-926a2473d0c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a105d0e9d4074f2ba1c131df2a5a9fe9)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:33.265960 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c5567d-0192-44c4-b157-926a2473d0c0 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:33.431594 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-05c5567d-0192-44c4-b157-926a2473d0c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a105d0e9d4074f2ba1c131df2a5a9fe9', 'request_id': 'req-05c5567d-0192-44c4-b157-926a2473d0c0'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:33.439237 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-05c5567d-0192-44c4-b157-926a2473d0c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a105d0e9d4074f2ba1c131df2a5a9fe9', 'request_id': 'req-05c5567d-0192-44c4-b157-926a2473d0c0'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:33.510493 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-05c5567d-0192-44c4-b157-926a2473d0c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a105d0e9d4074f2ba1c131df2a5a9fe9', 'request_id': 'req-05c5567d-0192-44c4-b157-926a2473d0c0'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:33.523107 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-05c5567d-0192-44c4-b157-926a2473d0c0 admin admin] The token cache is being invalidate because project a105d0e9d4074f2ba1c131df2a5a9fe9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:52:33.523402 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-05c5567d-0192-44c4-b157-926a2473d0c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:33.524268 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-05c5567d-0192-44c4-b157-926a2473d0c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:33.525885 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1100/2189] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:52:33 2022] DELETE /identity/v3/projects/a105d0e9d4074f2ba1c131df2a5a9fe9 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:33.530175 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70aaac2-6f34-484f-9f3d-406b8c29bc5b service neutron] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:33.560157 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70aaac2-6f34-484f-9f3d-406b8c29bc5b service neutron] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:33.674582 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db09a8c5-20a6-4264-b748-8dc2e21e417d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:33.676496 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db09a8c5-20a6-4264-b748-8dc2e21e417d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:33.676989 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db09a8c5-20a6-4264-b748-8dc2e21e417d admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:33.677521 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db09a8c5-20a6-4264-b748-8dc2e21e417d admin admin] PATH_INFO: `/v3/users/f287d8f2d0ff45ec86c2a0785b87a59b` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:33.680662 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db09a8c5-20a6-4264-b748-8dc2e21e417d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f287d8f2d0ff45ec86c2a0785b87a59b)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:33.683211 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db09a8c5-20a6-4264-b748-8dc2e21e417d admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:33.885411 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-db09a8c5-20a6-4264-b748-8dc2e21e417d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f287d8f2d0ff45ec86c2a0785b87a59b'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:33.886326 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-db09a8c5-20a6-4264-b748-8dc2e21e417d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f287d8f2d0ff45ec86c2a0785b87a59b'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:33.893707 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-db09a8c5-20a6-4264-b748-8dc2e21e417d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f287d8f2d0ff45ec86c2a0785b87a59b'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:33.894373 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-db09a8c5-20a6-4264-b748-8dc2e21e417d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f287d8f2d0ff45ec86c2a0785b87a59b'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:33.986178 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-db09a8c5-20a6-4264-b748-8dc2e21e417d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f287d8f2d0ff45ec86c2a0785b87a59b'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:34.022773 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1090/2190] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:52:33 2022] DELETE /identity/v3/users/f287d8f2d0ff45ec86c2a0785b87a59b => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:34.853351 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c5567d-0192-44c4-b157-926a2473d0c0 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:34.855748 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-05c5567d-0192-44c4-b157-926a2473d0c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:34.873766 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c5567d-0192-44c4-b157-926a2473d0c0 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:34.883978 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:34.959049 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8465d58-ea07-477f-8b0c-2567f9c1024e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:34.960943 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8465d58-ea07-477f-8b0c-2567f9c1024e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:34.965092 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8465d58-ea07-477f-8b0c-2567f9c1024e admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:34.965092 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8465d58-ea07-477f-8b0c-2567f9c1024e admin admin] PATH_INFO: `/v3/projects/bb64b47982674c6e8437e89f3fec355c` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:34.965770 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8465d58-ea07-477f-8b0c-2567f9c1024e admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb64b47982674c6e8437e89f3fec355c)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:34.969878 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8465d58-ea07-477f-8b0c-2567f9c1024e admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:35.062776 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db09a8c5-20a6-4264-b748-8dc2e21e417d admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:35.064816 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-db09a8c5-20a6-4264-b748-8dc2e21e417d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:35.079809 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db09a8c5-20a6-4264-b748-8dc2e21e417d admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:35.084754 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d600cb-4ae1-47c5-a70e-8aa1d94aff33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:35.085634 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d600cb-4ae1-47c5-a70e-8aa1d94aff33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:35.085934 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d600cb-4ae1-47c5-a70e-8aa1d94aff33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:35.086200 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d600cb-4ae1-47c5-a70e-8aa1d94aff33 admin admin] PATH_INFO: `/v3/users/34e6033285dd4ecfa61298e562000a49` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:35.087733 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d600cb-4ae1-47c5-a70e-8aa1d94aff33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34e6033285dd4ecfa61298e562000a49)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:35.089707 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d600cb-4ae1-47c5-a70e-8aa1d94aff33 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:35.114356 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:35.119593 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c8465d58-ea07-477f-8b0c-2567f9c1024e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb64b47982674c6e8437e89f3fec355c', 'request_id': 'req-c8465d58-ea07-477f-8b0c-2567f9c1024e'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:35.126352 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c8465d58-ea07-477f-8b0c-2567f9c1024e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb64b47982674c6e8437e89f3fec355c', 'request_id': 'req-c8465d58-ea07-477f-8b0c-2567f9c1024e'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:35.208811 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c8465d58-ea07-477f-8b0c-2567f9c1024e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb64b47982674c6e8437e89f3fec355c', 'request_id': 'req-c8465d58-ea07-477f-8b0c-2567f9c1024e'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:35.222902 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c8465d58-ea07-477f-8b0c-2567f9c1024e admin admin] The token cache is being invalidate because project bb64b47982674c6e8437e89f3fec355c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:52:35.223226 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c8465d58-ea07-477f-8b0c-2567f9c1024e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:35.223931 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c8465d58-ea07-477f-8b0c-2567f9c1024e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:35.225594 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1101/2191] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:52:34 2022] DELETE /identity/v3/projects/bb64b47982674c6e8437e89f3fec355c => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:35.232427 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6846a0cb-f48b-42a1-a4ca-2a9dd620647f None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:35.232698 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6846a0cb-f48b-42a1-a4ca-2a9dd620647f None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:35.233044 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6846a0cb-f48b-42a1-a4ca-2a9dd620647f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:35.257474 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-47d600cb-4ae1-47c5-a70e-8aa1d94aff33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34e6033285dd4ecfa61298e562000a49'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:35.259441 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-47d600cb-4ae1-47c5-a70e-8aa1d94aff33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34e6033285dd4ecfa61298e562000a49'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:35.263912 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-6846a0cb-f48b-42a1-a4ca-2a9dd620647f None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:35.265124 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-6846a0cb-f48b-42a1-a4ca-2a9dd620647f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:35.274830 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-47d600cb-4ae1-47c5-a70e-8aa1d94aff33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34e6033285dd4ecfa61298e562000a49'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:35.276444 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-47d600cb-4ae1-47c5-a70e-8aa1d94aff33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34e6033285dd4ecfa61298e562000a49'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:35.339752 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-47d600cb-4ae1-47c5-a70e-8aa1d94aff33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34e6033285dd4ecfa61298e562000a49'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:35.340950 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1102/2192] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:52:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:52:35.349734 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ab32fa-e518-4e95-a20f-976f63be7fcd None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:35.349962 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ab32fa-e518-4e95-a20f-976f63be7fcd None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:35.351862 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ab32fa-e518-4e95-a20f-976f63be7fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:35.366929 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1091/2193] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:52:35 2022] DELETE /identity/v3/users/34e6033285dd4ecfa61298e562000a49 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:52:35.373229 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-75ab32fa-e518-4e95-a20f-976f63be7fcd None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:35.373834 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d8730-95e7-4e9b-b288-d4701250f752 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:35.374286 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d8730-95e7-4e9b-b288-d4701250f752 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:35.374713 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d8730-95e7-4e9b-b288-d4701250f752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:35.376526 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-75ab32fa-e518-4e95-a20f-976f63be7fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:35.396602 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-070d8730-95e7-4e9b-b288-d4701250f752 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:35.397951 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-070d8730-95e7-4e9b-b288-d4701250f752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:35.484195 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1103/2194] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:52:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:35.485943 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ab32fa-e518-4e95-a20f-976f63be7fcd None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:35.497640 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ab32fa-e518-4e95-a20f-976f63be7fcd None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:35.499081 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1092/2195] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:52:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:35.505974 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37282b9f-aca2-411a-bc9a-7dcf1b589094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:35.506937 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37282b9f-aca2-411a-bc9a-7dcf1b589094 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:35.507381 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37282b9f-aca2-411a-bc9a-7dcf1b589094 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:35.507955 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37282b9f-aca2-411a-bc9a-7dcf1b589094 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:35.509072 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37282b9f-aca2-411a-bc9a-7dcf1b589094 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:52:35.509407 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37282b9f-aca2-411a-bc9a-7dcf1b589094 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:35.511551 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070d8730-95e7-4e9b-b288-d4701250f752 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:35.511722 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37282b9f-aca2-411a-bc9a-7dcf1b589094 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:35.513392 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-070d8730-95e7-4e9b-b288-d4701250f752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:35.529282 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1104/2196] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jun 15 14:52:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:52:35.530817 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37282b9f-aca2-411a-bc9a-7dcf1b589094 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:35.542074 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:35.551341 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37282b9f-aca2-411a-bc9a-7dcf1b589094 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:35.553731 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3393377-ea71-4d6a-8322-06477fffb6f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:35.554502 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3393377-ea71-4d6a-8322-06477fffb6f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:35.554848 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3393377-ea71-4d6a-8322-06477fffb6f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:35.555762 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3393377-ea71-4d6a-8322-06477fffb6f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:35.556826 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3393377-ea71-4d6a-8322-06477fffb6f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:35.558821 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3393377-ea71-4d6a-8322-06477fffb6f3 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:35.596430 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070d8730-95e7-4e9b-b288-d4701250f752 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:35.600342 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e868361-312a-4952-8d1e-eb314143d8f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:35.600996 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e868361-312a-4952-8d1e-eb314143d8f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:35.601426 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e868361-312a-4952-8d1e-eb314143d8f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:35.604261 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e868361-312a-4952-8d1e-eb314143d8f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:35.612071 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e868361-312a-4952-8d1e-eb314143d8f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:35.613910 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e868361-312a-4952-8d1e-eb314143d8f6 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:35.625064 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1093/2197] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:52:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:52:35.647023 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1105/2198] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:52:35 2022] POST /identity/v3/projects => generated 379 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:52:35.651633 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e868361-312a-4952-8d1e-eb314143d8f6 service neutron] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:35.661655 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e868361-312a-4952-8d1e-eb314143d8f6 service neutron] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:35.708301 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:35.726127 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc875969-58bd-4b44-b0aa-957fda91d2af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:35.727149 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc875969-58bd-4b44-b0aa-957fda91d2af admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:35.727603 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc875969-58bd-4b44-b0aa-957fda91d2af admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:35.728038 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc875969-58bd-4b44-b0aa-957fda91d2af admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:35.728983 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc875969-58bd-4b44-b0aa-957fda91d2af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:35.730922 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc875969-58bd-4b44-b0aa-957fda91d2af admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:35.813172 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1094/2199] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:52:35 2022] POST /identity/v3/users => generated 290 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:52:35.814662 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3393377-ea71-4d6a-8322-06477fffb6f3 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:35.826655 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3393377-ea71-4d6a-8322-06477fffb6f3 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:35.832309 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84e57f6-fb56-4845-b2a0-7b1c0a8b07d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:35.833036 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e57f6-fb56-4845-b2a0-7b1c0a8b07d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:35.833317 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e57f6-fb56-4845-b2a0-7b1c0a8b07d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:35.833592 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e57f6-fb56-4845-b2a0-7b1c0a8b07d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:35.834326 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84e57f6-fb56-4845-b2a0-7b1c0a8b07d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:35.835986 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84e57f6-fb56-4845-b2a0-7b1c0a8b07d2 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:35.846958 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1106/2200] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:52:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:52:35.852620 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc875969-58bd-4b44-b0aa-957fda91d2af admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:35.864891 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc875969-58bd-4b44-b0aa-957fda91d2af admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:35.870309 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f92d99-30b1-4269-9a2a-fdb7a2485a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:35.870978 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f92d99-30b1-4269-9a2a-fdb7a2485a18 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:35.871263 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f92d99-30b1-4269-9a2a-fdb7a2485a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:35.871539 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f92d99-30b1-4269-9a2a-fdb7a2485a18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:35.872303 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f92d99-30b1-4269-9a2a-fdb7a2485a18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:35.873922 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f92d99-30b1-4269-9a2a-fdb7a2485a18 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:35.889271 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1095/2201] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:52:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:52:35.890364 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84e57f6-fb56-4845-b2a0-7b1c0a8b07d2 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:35.900946 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84e57f6-fb56-4845-b2a0-7b1c0a8b07d2 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:35.905854 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d401cfd-e39f-4e87-98eb-4101f18519e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:35.906770 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d401cfd-e39f-4e87-98eb-4101f18519e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:35.907043 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d401cfd-e39f-4e87-98eb-4101f18519e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:35.907313 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d401cfd-e39f-4e87-98eb-4101f18519e7 admin admin] PATH_INFO: `/v3/projects/e9ba56650d2042b9ade9820a18566b1e/users/891c242c20ca4f23bd0db8b7cf6189cb/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:35.923186 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d401cfd-e39f-4e87-98eb-4101f18519e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9ba56650d2042b9ade9820a18566b1e, user_id=891c242c20ca4f23bd0db8b7cf6189cb, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:35.925867 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d401cfd-e39f-4e87-98eb-4101f18519e7 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:35.985215 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1107/2202] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:52:35 2022] PUT /identity/v3/projects/e9ba56650d2042b9ade9820a18566b1e/users/891c242c20ca4f23bd0db8b7cf6189cb/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:52:35.995624 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f39400-7250-461c-b088-42f7e4cff8a3 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:35.995993 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f39400-7250-461c-b088-42f7e4cff8a3 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:35.996517 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f39400-7250-461c-b088-42f7e4cff8a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:36.023416 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-a9f39400-7250-461c-b088-42f7e4cff8a3 None None] MFA Rules not processed for user `891c242c20ca4f23bd0db8b7cf6189cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:36.025160 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-a9f39400-7250-461c-b088-42f7e4cff8a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:36.048629 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d401cfd-e39f-4e87-98eb-4101f18519e7 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:36.053428 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-6d401cfd-e39f-4e87-98eb-4101f18519e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:36.094820 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1096/2203] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:52:35 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:36.099130 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f39400-7250-461c-b088-42f7e4cff8a3 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:36.128179 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d401cfd-e39f-4e87-98eb-4101f18519e7 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:36.133771 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ac8e40-1420-4c64-b652-4e5411ad30e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:36.134668 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ac8e40-1420-4c64-b652-4e5411ad30e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:36.135015 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ac8e40-1420-4c64-b652-4e5411ad30e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:36.135358 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ac8e40-1420-4c64-b652-4e5411ad30e3 admin admin] PATH_INFO: `/v3/projects/93af4a5c7c424941a435e44272d3f19f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:36.136949 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ac8e40-1420-4c64-b652-4e5411ad30e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93af4a5c7c424941a435e44272d3f19f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:36.138838 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ac8e40-1420-4c64-b652-4e5411ad30e3 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:36.164159 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f39400-7250-461c-b088-42f7e4cff8a3 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:36.169132 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df27abf8-5a0a-40fa-a9cf-d3b142cd55db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:36.169797 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df27abf8-5a0a-40fa-a9cf-d3b142cd55db admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:36.170083 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df27abf8-5a0a-40fa-a9cf-d3b142cd55db admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:36.170361 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df27abf8-5a0a-40fa-a9cf-d3b142cd55db admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:36.171056 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df27abf8-5a0a-40fa-a9cf-d3b142cd55db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:36.173005 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df27abf8-5a0a-40fa-a9cf-d3b142cd55db admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:36.267591 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-82ac8e40-1420-4c64-b652-4e5411ad30e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93af4a5c7c424941a435e44272d3f19f', 'request_id': 'req-82ac8e40-1420-4c64-b652-4e5411ad30e3'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:36.270098 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1097/2204] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:52:36 2022] POST /identity/v3/projects => generated 381 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:52:36.274758 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df27abf8-5a0a-40fa-a9cf-d3b142cd55db admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:36.276258 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-82ac8e40-1420-4c64-b652-4e5411ad30e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93af4a5c7c424941a435e44272d3f19f', 'request_id': 'req-82ac8e40-1420-4c64-b652-4e5411ad30e3'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:36.314857 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:36.335702 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df27abf8-5a0a-40fa-a9cf-d3b142cd55db admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:36.340979 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f1db1d-177c-4806-9c24-02a7fcd30250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:36.341635 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f1db1d-177c-4806-9c24-02a7fcd30250 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:36.341914 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f1db1d-177c-4806-9c24-02a7fcd30250 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:36.342190 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f1db1d-177c-4806-9c24-02a7fcd30250 admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:36.342941 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f1db1d-177c-4806-9c24-02a7fcd30250 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:36.344873 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f1db1d-177c-4806-9c24-02a7fcd30250 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:36.349670 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-82ac8e40-1420-4c64-b652-4e5411ad30e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93af4a5c7c424941a435e44272d3f19f', 'request_id': 'req-82ac8e40-1420-4c64-b652-4e5411ad30e3'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:36.361678 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-82ac8e40-1420-4c64-b652-4e5411ad30e3 admin admin] The token cache is being invalidate because project 93af4a5c7c424941a435e44272d3f19f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:52:36.361964 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-82ac8e40-1420-4c64-b652-4e5411ad30e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:36.362640 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-82ac8e40-1420-4c64-b652-4e5411ad30e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:36.364439 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1108/2205] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:52:36 2022] DELETE /identity/v3/projects/93af4a5c7c424941a435e44272d3f19f => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:52:36.367998 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ac8e40-1420-4c64-b652-4e5411ad30e3 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:36.369331 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-82ac8e40-1420-4c64-b652-4e5411ad30e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:36.374189 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:36.434077 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ac8e40-1420-4c64-b652-4e5411ad30e3 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:36.440941 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1098/2206] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:52:36 2022] POST /identity/v3/users => generated 297 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:52:36.442020 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f1db1d-177c-4806-9c24-02a7fcd30250 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:36.446441 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9c3c63-f546-4fc8-a373-f416f27576de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:36.446441 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c3c63-f546-4fc8-a373-f416f27576de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:36.446441 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c3c63-f546-4fc8-a373-f416f27576de admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:36.446441 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c3c63-f546-4fc8-a373-f416f27576de admin admin] PATH_INFO: `/v3/users/81dd0abdb9b14ee880f1c652f53e737d` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:36.447023 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-08f1db1d-177c-4806-9c24-02a7fcd30250 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:36.447118 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9c3c63-f546-4fc8-a373-f416f27576de admin admin] RBAC: Authorizing `identity:delete_user(user_id=81dd0abdb9b14ee880f1c652f53e737d)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:36.448360 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9c3c63-f546-4fc8-a373-f416f27576de admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:36.458542 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f1db1d-177c-4806-9c24-02a7fcd30250 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:36.464588 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa968049-f58b-4c55-b92e-60b7e0656ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:36.465560 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa968049-f58b-4c55-b92e-60b7e0656ea6 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:36.465912 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa968049-f58b-4c55-b92e-60b7e0656ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:36.468273 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa968049-f58b-4c55-b92e-60b7e0656ea6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:36.470712 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa968049-f58b-4c55-b92e-60b7e0656ea6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:36.470712 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa968049-f58b-4c55-b92e-60b7e0656ea6 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:36.483596 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1099/2207] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:52:36 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:52:36.487593 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa968049-f58b-4c55-b92e-60b7e0656ea6 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:36.500529 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa968049-f58b-4c55-b92e-60b7e0656ea6 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:36.505991 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bf3b9d-a9f7-488b-ae32-9fbec22dbb41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:36.507049 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf3b9d-a9f7-488b-ae32-9fbec22dbb41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:36.507364 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf3b9d-a9f7-488b-ae32-9fbec22dbb41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:36.507709 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf3b9d-a9f7-488b-ae32-9fbec22dbb41 admin admin] PATH_INFO: `/v3/projects/b454d08f2a7c4b0f84f19d8d6adb9bed/users/bdf2f519dc8744869c7bd7ad0585b120/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:36.526418 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bf3b9d-a9f7-488b-ae32-9fbec22dbb41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b454d08f2a7c4b0f84f19d8d6adb9bed, user_id=bdf2f519dc8744869c7bd7ad0585b120, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:36.532150 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bf3b9d-a9f7-488b-ae32-9fbec22dbb41 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:36.643355 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1100/2208] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:52:36 2022] PUT /identity/v3/projects/b454d08f2a7c4b0f84f19d8d6adb9bed/users/bdf2f519dc8744869c7bd7ad0585b120/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:36.647560 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bf3b9d-a9f7-488b-ae32-9fbec22dbb41 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:36.654060 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ad9c3c63-f546-4fc8-a373-f416f27576de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81dd0abdb9b14ee880f1c652f53e737d'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:36.654837 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ad9c3c63-f546-4fc8-a373-f416f27576de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81dd0abdb9b14ee880f1c652f53e737d'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:36.655436 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ad9c3c63-f546-4fc8-a373-f416f27576de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81dd0abdb9b14ee880f1c652f53e737d'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:36.711153 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bf3b9d-a9f7-488b-ae32-9fbec22dbb41 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:36.716351 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe78359-8d88-4968-8aa4-48ab480ce2ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:36.719116 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe78359-8d88-4968-8aa4-48ab480ce2ba admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:36.719116 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe78359-8d88-4968-8aa4-48ab480ce2ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:36.719116 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe78359-8d88-4968-8aa4-48ab480ce2ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:36.719116 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe78359-8d88-4968-8aa4-48ab480ce2ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:36.719965 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe78359-8d88-4968-8aa4-48ab480ce2ba admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:36.732083 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1101/2209] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:52:36 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:52:36.735577 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe78359-8d88-4968-8aa4-48ab480ce2ba admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:36.737426 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-dfe78359-8d88-4968-8aa4-48ab480ce2ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:36.738780 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ad9c3c63-f546-4fc8-a373-f416f27576de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81dd0abdb9b14ee880f1c652f53e737d'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:36.754304 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ad9c3c63-f546-4fc8-a373-f416f27576de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81dd0abdb9b14ee880f1c652f53e737d'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:36.757019 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe78359-8d88-4968-8aa4-48ab480ce2ba admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:36.763754 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97be8d9-fc98-46c4-9e41-2639e5a8bfc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:36.764767 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97be8d9-fc98-46c4-9e41-2639e5a8bfc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:36.765047 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97be8d9-fc98-46c4-9e41-2639e5a8bfc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:36.765323 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97be8d9-fc98-46c4-9e41-2639e5a8bfc5 admin admin] PATH_INFO: `/v3/projects/b454d08f2a7c4b0f84f19d8d6adb9bed/users/bdf2f519dc8744869c7bd7ad0585b120/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:36.767393 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97be8d9-fc98-46c4-9e41-2639e5a8bfc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b454d08f2a7c4b0f84f19d8d6adb9bed, user_id=bdf2f519dc8744869c7bd7ad0585b120, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:36.772374 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1109/2210] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:52:36 2022] DELETE /identity/v3/users/81dd0abdb9b14ee880f1c652f53e737d => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:52:36.772566 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97be8d9-fc98-46c4-9e41-2639e5a8bfc5 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:36.787232 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1102/2211] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:52:36 2022] PUT /identity/v3/projects/b454d08f2a7c4b0f84f19d8d6adb9bed/users/bdf2f519dc8744869c7bd7ad0585b120/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:52:36.798689 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805d0061-411d-4e0e-ab89-f0ecb4952f31 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:36.798975 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805d0061-411d-4e0e-ab89-f0ecb4952f31 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:36.799250 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805d0061-411d-4e0e-ab89-f0ecb4952f31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:36.836419 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-805d0061-411d-4e0e-ab89-f0ecb4952f31 None None] MFA Rules not processed for user `bdf2f519dc8744869c7bd7ad0585b120`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:36.837717 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-805d0061-411d-4e0e-ab89-f0ecb4952f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:36.926543 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1110/2212] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:52:36 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:36.938997 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97be8d9-fc98-46c4-9e41-2639e5a8bfc5 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:36.941692 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-c97be8d9-fc98-46c4-9e41-2639e5a8bfc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:37.058896 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97be8d9-fc98-46c4-9e41-2639e5a8bfc5 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:37.062845 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bde007-269e-4a92-afdf-e6abfbbd0bdf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:37.066087 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bde007-269e-4a92-afdf-e6abfbbd0bdf service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:37.066087 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bde007-269e-4a92-afdf-e6abfbbd0bdf service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:37.066087 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bde007-269e-4a92-afdf-e6abfbbd0bdf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:37.075797 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bde007-269e-4a92-afdf-e6abfbbd0bdf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:37.078197 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bde007-269e-4a92-afdf-e6abfbbd0bdf service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:37.083554 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:37.091141 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1103/2213] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:52:36 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:52:37.261990 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805d0061-411d-4e0e-ab89-f0ecb4952f31 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:37.263985 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-805d0061-411d-4e0e-ab89-f0ecb4952f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:37.286897 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:37.372945 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805d0061-411d-4e0e-ab89-f0ecb4952f31 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:37.379736 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2d8473-b794-4c30-87c0-cf80a3619cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:37.380677 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2d8473-b794-4c30-87c0-cf80a3619cc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:37.380966 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2d8473-b794-4c30-87c0-cf80a3619cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:37.381244 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2d8473-b794-4c30-87c0-cf80a3619cc8 admin admin] PATH_INFO: `/v3/projects/757549c6346a41ac8960423009ad3524` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:37.382456 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2d8473-b794-4c30-87c0-cf80a3619cc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=757549c6346a41ac8960423009ad3524)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:37.385048 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2d8473-b794-4c30-87c0-cf80a3619cc8 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:37.505195 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5b2d8473-b794-4c30-87c0-cf80a3619cc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '757549c6346a41ac8960423009ad3524', 'request_id': 'req-5b2d8473-b794-4c30-87c0-cf80a3619cc8'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:37.516279 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5b2d8473-b794-4c30-87c0-cf80a3619cc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '757549c6346a41ac8960423009ad3524', 'request_id': 'req-5b2d8473-b794-4c30-87c0-cf80a3619cc8'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:37.612603 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5b2d8473-b794-4c30-87c0-cf80a3619cc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '757549c6346a41ac8960423009ad3524', 'request_id': 'req-5b2d8473-b794-4c30-87c0-cf80a3619cc8'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:37.633805 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5b2d8473-b794-4c30-87c0-cf80a3619cc8 admin admin] The token cache is being invalidate because project 757549c6346a41ac8960423009ad3524 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:52:37.633805 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5b2d8473-b794-4c30-87c0-cf80a3619cc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:37.633805 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5b2d8473-b794-4c30-87c0-cf80a3619cc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:37.635597 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1111/2214] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:52:37 2022] DELETE /identity/v3/projects/757549c6346a41ac8960423009ad3524 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:37.640278 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bde007-269e-4a92-afdf-e6abfbbd0bdf service cinder] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:37.652250 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bde007-269e-4a92-afdf-e6abfbbd0bdf service cinder] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:37.733854 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71968cb9-f97b-4942-aa15-6ea31a0d749d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:37.735797 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71968cb9-f97b-4942-aa15-6ea31a0d749d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:37.736259 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71968cb9-f97b-4942-aa15-6ea31a0d749d admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:37.736680 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71968cb9-f97b-4942-aa15-6ea31a0d749d admin admin] PATH_INFO: `/v3/users/6a5affc371574927b149409da8df2670` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:37.739826 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71968cb9-f97b-4942-aa15-6ea31a0d749d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a5affc371574927b149409da8df2670)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:37.743372 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71968cb9-f97b-4942-aa15-6ea31a0d749d admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:37.765672 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2d8473-b794-4c30-87c0-cf80a3619cc8 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:37.767361 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-5b2d8473-b794-4c30-87c0-cf80a3619cc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:37.793329 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2d8473-b794-4c30-87c0-cf80a3619cc8 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:37.798532 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb993c8-82ed-4f4b-91b7-92deaf506923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:37.799429 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb993c8-82ed-4f4b-91b7-92deaf506923 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:37.799715 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb993c8-82ed-4f4b-91b7-92deaf506923 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:37.799998 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb993c8-82ed-4f4b-91b7-92deaf506923 admin admin] PATH_INFO: `/v3/users/891c242c20ca4f23bd0db8b7cf6189cb` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:37.801326 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb993c8-82ed-4f4b-91b7-92deaf506923 admin admin] RBAC: Authorizing `identity:delete_user(user_id=891c242c20ca4f23bd0db8b7cf6189cb)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:37.803895 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb993c8-82ed-4f4b-91b7-92deaf506923 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:37.995839 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-71968cb9-f97b-4942-aa15-6ea31a0d749d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a5affc371574927b149409da8df2670'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:37.996580 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-71968cb9-f97b-4942-aa15-6ea31a0d749d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a5affc371574927b149409da8df2670'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:38.004459 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-71968cb9-f97b-4942-aa15-6ea31a0d749d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a5affc371574927b149409da8df2670'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:38.005210 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-71968cb9-f97b-4942-aa15-6ea31a0d749d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a5affc371574927b149409da8df2670'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:38.027391 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-acb993c8-82ed-4f4b-91b7-92deaf506923 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '891c242c20ca4f23bd0db8b7cf6189cb'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:38.028561 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-acb993c8-82ed-4f4b-91b7-92deaf506923 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '891c242c20ca4f23bd0db8b7cf6189cb'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:38.029335 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-acb993c8-82ed-4f4b-91b7-92deaf506923 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '891c242c20ca4f23bd0db8b7cf6189cb'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:38.083113 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-71968cb9-f97b-4942-aa15-6ea31a0d749d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a5affc371574927b149409da8df2670'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:38.106601 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1104/2215] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:52:37 2022] DELETE /identity/v3/users/6a5affc371574927b149409da8df2670 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:38.140310 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-acb993c8-82ed-4f4b-91b7-92deaf506923 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '891c242c20ca4f23bd0db8b7cf6189cb'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:38.159285 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-acb993c8-82ed-4f4b-91b7-92deaf506923 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '891c242c20ca4f23bd0db8b7cf6189cb'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:38.175866 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1112/2216] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:52:37 2022] DELETE /identity/v3/users/891c242c20ca4f23bd0db8b7cf6189cb => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:52:38.181824 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc80ef6-0dd6-49c5-a9f2-bdb4fa3aaf4e None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:38.182180 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc80ef6-0dd6-49c5-a9f2-bdb4fa3aaf4e None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:38.182477 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc80ef6-0dd6-49c5-a9f2-bdb4fa3aaf4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:38.201865 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-1bc80ef6-0dd6-49c5-a9f2-bdb4fa3aaf4e None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:38.203087 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-1bc80ef6-0dd6-49c5-a9f2-bdb4fa3aaf4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:38.278492 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1105/2217] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:52:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:38.290732 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb993c8-82ed-4f4b-91b7-92deaf506923 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:38.293148 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-acb993c8-82ed-4f4b-91b7-92deaf506923 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:38.315652 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:38.394728 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb993c8-82ed-4f4b-91b7-92deaf506923 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:38.399502 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca3e0a8-93bf-4d0b-84cf-686e504b54ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:38.400844 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca3e0a8-93bf-4d0b-84cf-686e504b54ef service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:38.401330 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca3e0a8-93bf-4d0b-84cf-686e504b54ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:38.401330 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca3e0a8-93bf-4d0b-84cf-686e504b54ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:38.418455 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca3e0a8-93bf-4d0b-84cf-686e504b54ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:38.421408 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca3e0a8-93bf-4d0b-84cf-686e504b54ef service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:38.432830 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1113/2218] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:52:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:52:38.604544 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc80ef6-0dd6-49c5-a9f2-bdb4fa3aaf4e None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:38.606558 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-1bc80ef6-0dd6-49c5-a9f2-bdb4fa3aaf4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:38.630691 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc80ef6-0dd6-49c5-a9f2-bdb4fa3aaf4e None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:38.637623 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec3ec5c-4c54-45da-9460-b73d1711a80f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:38.638586 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec3ec5c-4c54-45da-9460-b73d1711a80f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:38.639003 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec3ec5c-4c54-45da-9460-b73d1711a80f admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:38.639478 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec3ec5c-4c54-45da-9460-b73d1711a80f admin admin] PATH_INFO: `/v3/projects/02115930a70c435f88324970dcc3c3e9` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:38.641485 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec3ec5c-4c54-45da-9460-b73d1711a80f admin admin] RBAC: Authorizing `identity:delete_project(project_id=02115930a70c435f88324970dcc3c3e9)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:38.646162 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec3ec5c-4c54-45da-9460-b73d1711a80f admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:38.670988 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:38.792411 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-bec3ec5c-4c54-45da-9460-b73d1711a80f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02115930a70c435f88324970dcc3c3e9', 'request_id': 'req-bec3ec5c-4c54-45da-9460-b73d1711a80f'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:38.894755 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-bec3ec5c-4c54-45da-9460-b73d1711a80f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02115930a70c435f88324970dcc3c3e9', 'request_id': 'req-bec3ec5c-4c54-45da-9460-b73d1711a80f'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:38.904602 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-bec3ec5c-4c54-45da-9460-b73d1711a80f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02115930a70c435f88324970dcc3c3e9', 'request_id': 'req-bec3ec5c-4c54-45da-9460-b73d1711a80f'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:38.919034 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-bec3ec5c-4c54-45da-9460-b73d1711a80f admin admin] The token cache is being invalidate because project 02115930a70c435f88324970dcc3c3e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:52:38.919352 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-bec3ec5c-4c54-45da-9460-b73d1711a80f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:38.920257 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-bec3ec5c-4c54-45da-9460-b73d1711a80f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:38.938911 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1106/2219] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:52:38 2022] DELETE /identity/v3/projects/02115930a70c435f88324970dcc3c3e9 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:38.941225 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a402934-76a9-4c8e-8fe4-92afe2f9c666 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:38.941738 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a402934-76a9-4c8e-8fe4-92afe2f9c666 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:38.942262 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a402934-76a9-4c8e-8fe4-92afe2f9c666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:38.971896 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-7a402934-76a9-4c8e-8fe4-92afe2f9c666 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:38.973829 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-7a402934-76a9-4c8e-8fe4-92afe2f9c666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:39.065282 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1114/2220] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:52:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:39.076505 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c3ab06-45d2-4571-a14b-d77005666d23 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:39.076938 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c3ab06-45d2-4571-a14b-d77005666d23 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:39.077404 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c3ab06-45d2-4571-a14b-d77005666d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:39.097259 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a402934-76a9-4c8e-8fe4-92afe2f9c666 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:39.099853 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-7a402934-76a9-4c8e-8fe4-92afe2f9c666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:39.113858 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-70c3ab06-45d2-4571-a14b-d77005666d23 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:39.115486 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-70c3ab06-45d2-4571-a14b-d77005666d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:39.115968 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a402934-76a9-4c8e-8fe4-92afe2f9c666 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:39.130246 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1107/2221] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:52:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:39.136286 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c3ab06-45d2-4571-a14b-d77005666d23 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:39.148812 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad81178b-2df7-4a22-a094-f9f1660f1676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:39.150297 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c3ab06-45d2-4571-a14b-d77005666d23 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:39.152916 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad81178b-2df7-4a22-a094-f9f1660f1676 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:39.153685 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad81178b-2df7-4a22-a094-f9f1660f1676 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:39.154189 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad81178b-2df7-4a22-a094-f9f1660f1676 admin admin] PATH_INFO: `/v3/projects/e9ba56650d2042b9ade9820a18566b1e` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:39.156613 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad81178b-2df7-4a22-a094-f9f1660f1676 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9ba56650d2042b9ade9820a18566b1e)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:39.157092 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1585469-b1af-4d97-8d4b-5542e8d4969b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:39.158030 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1585469-b1af-4d97-8d4b-5542e8d4969b admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:39.158464 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1585469-b1af-4d97-8d4b-5542e8d4969b admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:39.158949 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1585469-b1af-4d97-8d4b-5542e8d4969b admin admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:39.159343 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad81178b-2df7-4a22-a094-f9f1660f1676 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:39.159838 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1585469-b1af-4d97-8d4b-5542e8d4969b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:52:39.160179 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1585469-b1af-4d97-8d4b-5542e8d4969b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:39.162037 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1585469-b1af-4d97-8d4b-5542e8d4969b admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:39.188711 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1108/2222] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jun 15 14:52:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:52:39.191363 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1585469-b1af-4d97-8d4b-5542e8d4969b admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:39.204290 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1585469-b1af-4d97-8d4b-5542e8d4969b admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:39.209552 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecad9c14-74c3-4071-b0ae-ce0b34062a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:39.210270 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecad9c14-74c3-4071-b0ae-ce0b34062a74 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:39.210559 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecad9c14-74c3-4071-b0ae-ce0b34062a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:39.210837 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecad9c14-74c3-4071-b0ae-ce0b34062a74 admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:39.212279 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecad9c14-74c3-4071-b0ae-ce0b34062a74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:39.214493 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecad9c14-74c3-4071-b0ae-ce0b34062a74 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:39.300903 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ad81178b-2df7-4a22-a094-f9f1660f1676 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9ba56650d2042b9ade9820a18566b1e', 'request_id': 'req-ad81178b-2df7-4a22-a094-f9f1660f1676'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:39.308727 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ad81178b-2df7-4a22-a094-f9f1660f1676 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9ba56650d2042b9ade9820a18566b1e', 'request_id': 'req-ad81178b-2df7-4a22-a094-f9f1660f1676'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:39.370281 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ad81178b-2df7-4a22-a094-f9f1660f1676 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9ba56650d2042b9ade9820a18566b1e', 'request_id': 'req-ad81178b-2df7-4a22-a094-f9f1660f1676'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:39.385019 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ad81178b-2df7-4a22-a094-f9f1660f1676 admin admin] The token cache is being invalidate because project e9ba56650d2042b9ade9820a18566b1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:52:39.385620 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ad81178b-2df7-4a22-a094-f9f1660f1676 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:39.386835 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ad81178b-2df7-4a22-a094-f9f1660f1676 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:39.390330 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1109/2223] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:52:39 2022] POST /identity/v3/projects => generated 377 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:52:39.390615 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1115/2224] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:52:39 2022] DELETE /identity/v3/projects/e9ba56650d2042b9ade9820a18566b1e => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:39.395349 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad81178b-2df7-4a22-a094-f9f1660f1676 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:39.397881 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-ad81178b-2df7-4a22-a094-f9f1660f1676 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:39.400038 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecad9c14-74c3-4071-b0ae-ce0b34062a74 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:39.401826 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-ecad9c14-74c3-4071-b0ae-ce0b34062a74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:39.474136 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:39.513031 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecad9c14-74c3-4071-b0ae-ce0b34062a74 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:39.518549 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad81178b-2df7-4a22-a094-f9f1660f1676 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:39.523119 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4ea687-f31f-428b-92c7-8b27b7d68c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:39.524648 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ea687-f31f-428b-92c7-8b27b7d68c40 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:39.525037 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ea687-f31f-428b-92c7-8b27b7d68c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:39.525425 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ea687-f31f-428b-92c7-8b27b7d68c40 admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:39.525590 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c16d515-ddba-4b98-ba5a-ecd869cfb5a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:39.526449 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4ea687-f31f-428b-92c7-8b27b7d68c40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:39.526628 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c16d515-ddba-4b98-ba5a-ecd869cfb5a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:39.526778 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c16d515-ddba-4b98-ba5a-ecd869cfb5a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:39.527064 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c16d515-ddba-4b98-ba5a-ecd869cfb5a8 admin admin] PATH_INFO: `/v3/users/bdf2f519dc8744869c7bd7ad0585b120` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:39.528417 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c16d515-ddba-4b98-ba5a-ecd869cfb5a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdf2f519dc8744869c7bd7ad0585b120)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:39.529365 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4ea687-f31f-428b-92c7-8b27b7d68c40 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:39.530600 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c16d515-ddba-4b98-ba5a-ecd869cfb5a8 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:39.645445 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1110/2225] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:52:39 2022] POST /identity/v3/users => generated 289 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:52:39.650510 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4ea687-f31f-428b-92c7-8b27b7d68c40 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:39.660683 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4ea687-f31f-428b-92c7-8b27b7d68c40 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:39.666325 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721c3a84-c80a-4af9-a776-1b9028b3fc18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:39.669161 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721c3a84-c80a-4af9-a776-1b9028b3fc18 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:39.669161 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721c3a84-c80a-4af9-a776-1b9028b3fc18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:39.669161 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721c3a84-c80a-4af9-a776-1b9028b3fc18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:39.669161 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721c3a84-c80a-4af9-a776-1b9028b3fc18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:39.669858 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721c3a84-c80a-4af9-a776-1b9028b3fc18 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:39.680853 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1111/2226] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:52:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:52:39.685681 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721c3a84-c80a-4af9-a776-1b9028b3fc18 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:39.696837 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721c3a84-c80a-4af9-a776-1b9028b3fc18 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:39.702689 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1338cc2f-0659-4d51-a087-978ab9972be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:39.703562 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1338cc2f-0659-4d51-a087-978ab9972be6 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:39.704017 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1338cc2f-0659-4d51-a087-978ab9972be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:39.704475 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1338cc2f-0659-4d51-a087-978ab9972be6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:39.705426 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1338cc2f-0659-4d51-a087-978ab9972be6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:39.707655 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1338cc2f-0659-4d51-a087-978ab9972be6 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:39.722642 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1112/2227] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:52:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:52:39.723835 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1338cc2f-0659-4d51-a087-978ab9972be6 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:39.735856 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1338cc2f-0659-4d51-a087-978ab9972be6 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:39.740583 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02bef4d-1109-41e7-a26a-14e1498ecd38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:39.741478 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02bef4d-1109-41e7-a26a-14e1498ecd38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:39.741755 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02bef4d-1109-41e7-a26a-14e1498ecd38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:39.742032 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02bef4d-1109-41e7-a26a-14e1498ecd38 admin admin] PATH_INFO: `/v3/projects/f5511d18ecfb4e32a6398261be03a447/users/5826763f3c24400d9016394999717e50/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:39.757800 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02bef4d-1109-41e7-a26a-14e1498ecd38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5511d18ecfb4e32a6398261be03a447, user_id=5826763f3c24400d9016394999717e50, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:39.760567 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02bef4d-1109-41e7-a26a-14e1498ecd38 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:39.811828 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-6c16d515-ddba-4b98-ba5a-ecd869cfb5a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdf2f519dc8744869c7bd7ad0585b120'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:39.812497 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-6c16d515-ddba-4b98-ba5a-ecd869cfb5a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdf2f519dc8744869c7bd7ad0585b120'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:39.813173 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-6c16d515-ddba-4b98-ba5a-ecd869cfb5a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdf2f519dc8744869c7bd7ad0585b120'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:39.838039 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1113/2228] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:52:39 2022] PUT /identity/v3/projects/f5511d18ecfb4e32a6398261be03a447/users/5826763f3c24400d9016394999717e50/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:52:39.849867 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe14c57-8a90-414e-ba2f-f7edbbd6480e None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:39.850145 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe14c57-8a90-414e-ba2f-f7edbbd6480e None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:39.850424 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe14c57-8a90-414e-ba2f-f7edbbd6480e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:39.857118 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:INSERT=6 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:39.877396 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-6c16d515-ddba-4b98-ba5a-ecd869cfb5a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdf2f519dc8744869c7bd7ad0585b120'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:39.887307 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-8fe14c57-8a90-414e-ba2f-f7edbbd6480e None None] MFA Rules not processed for user `5826763f3c24400d9016394999717e50`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:39.888583 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-8fe14c57-8a90-414e-ba2f-f7edbbd6480e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:39.894883 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-6c16d515-ddba-4b98-ba5a-ecd869cfb5a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdf2f519dc8744869c7bd7ad0585b120'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:39.910516 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1116/2229] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:52:39 2022] DELETE /identity/v3/users/bdf2f519dc8744869c7bd7ad0585b120 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:39.974388 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1114/2230] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:52:39 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:52:39.986732 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c16d515-ddba-4b98-ba5a-ecd869cfb5a8 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:39.988602 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-6c16d515-ddba-4b98-ba5a-ecd869cfb5a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:40.067073 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c16d515-ddba-4b98-ba5a-ecd869cfb5a8 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:40.070416 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee56080b-8533-418a-b1cc-3e263799db8d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f18c57f27784694935bfb0bcea48a78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:40.072140 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee56080b-8533-418a-b1cc-3e263799db8d service glance] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:40.072626 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee56080b-8533-418a-b1cc-3e263799db8d service glance] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:40.073022 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee56080b-8533-418a-b1cc-3e263799db8d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:40.158159 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee56080b-8533-418a-b1cc-3e263799db8d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:40.159827 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee56080b-8533-418a-b1cc-3e263799db8d service glance] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:40.171590 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1117/2231] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:52:39 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:52:40.626842 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe14c57-8a90-414e-ba2f-f7edbbd6480e None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:40.628443 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-8fe14c57-8a90-414e-ba2f-f7edbbd6480e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:40.749238 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe14c57-8a90-414e-ba2f-f7edbbd6480e None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:40.824042 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cdb7b3-14e5-4f91-81f7-5ae2f653b719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:40.824940 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cdb7b3-14e5-4f91-81f7-5ae2f653b719 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:40.825226 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cdb7b3-14e5-4f91-81f7-5ae2f653b719 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:40.825500 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cdb7b3-14e5-4f91-81f7-5ae2f653b719 admin admin] PATH_INFO: `/v3/projects/b454d08f2a7c4b0f84f19d8d6adb9bed` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:40.826685 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cdb7b3-14e5-4f91-81f7-5ae2f653b719 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b454d08f2a7c4b0f84f19d8d6adb9bed)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:40.828631 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cdb7b3-14e5-4f91-81f7-5ae2f653b719 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:40.972875 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:40.979691 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-36cdb7b3-14e5-4f91-81f7-5ae2f653b719 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b454d08f2a7c4b0f84f19d8d6adb9bed', 'request_id': 'req-36cdb7b3-14e5-4f91-81f7-5ae2f653b719'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:41.069912 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-36cdb7b3-14e5-4f91-81f7-5ae2f653b719 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b454d08f2a7c4b0f84f19d8d6adb9bed', 'request_id': 'req-36cdb7b3-14e5-4f91-81f7-5ae2f653b719'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:41.078247 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-36cdb7b3-14e5-4f91-81f7-5ae2f653b719 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b454d08f2a7c4b0f84f19d8d6adb9bed', 'request_id': 'req-36cdb7b3-14e5-4f91-81f7-5ae2f653b719'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:41.095741 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-36cdb7b3-14e5-4f91-81f7-5ae2f653b719 admin admin] The token cache is being invalidate because project b454d08f2a7c4b0f84f19d8d6adb9bed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:52:41.096140 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-36cdb7b3-14e5-4f91-81f7-5ae2f653b719 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:41.097374 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-36cdb7b3-14e5-4f91-81f7-5ae2f653b719 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:41.102132 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1115/2232] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:52:40 2022] DELETE /identity/v3/projects/b454d08f2a7c4b0f84f19d8d6adb9bed => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:41.108647 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527066fe-224f-475e-9b6e-2c5f1f554a19 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:41.109271 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527066fe-224f-475e-9b6e-2c5f1f554a19 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:41.109271 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527066fe-224f-475e-9b6e-2c5f1f554a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:41.134956 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-527066fe-224f-475e-9b6e-2c5f1f554a19 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:41.136440 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-527066fe-224f-475e-9b6e-2c5f1f554a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:41.142087 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:41.215788 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1118/2233] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:52:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:41.218128 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a45b9f-ec9a-4483-8568-928d68522d4c None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:41.218484 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a45b9f-ec9a-4483-8568-928d68522d4c None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:41.218934 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a45b9f-ec9a-4483-8568-928d68522d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:41.240441 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-e1a45b9f-ec9a-4483-8568-928d68522d4c None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:41.243738 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-e1a45b9f-ec9a-4483-8568-928d68522d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:41.252813 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1116/2234] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:52:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:41.257597 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527066fe-224f-475e-9b6e-2c5f1f554a19 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:41.269224 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527066fe-224f-475e-9b6e-2c5f1f554a19 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:41.274630 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ff6948-b66b-46ab-84aa-58b74eae9b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:41.274789 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ff6948-b66b-46ab-84aa-58b74eae9b1c admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:41.275261 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ff6948-b66b-46ab-84aa-58b74eae9b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:41.275544 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ff6948-b66b-46ab-84aa-58b74eae9b1c admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:41.276331 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ff6948-b66b-46ab-84aa-58b74eae9b1c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:52:41.276645 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ff6948-b66b-46ab-84aa-58b74eae9b1c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:41.278266 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ff6948-b66b-46ab-84aa-58b74eae9b1c admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:41.292341 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1119/2235] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jun 15 14:52:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:52:41.297424 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a45b9f-ec9a-4483-8568-928d68522d4c None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:41.310416 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a45b9f-ec9a-4483-8568-928d68522d4c None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:41.319278 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25a1bb2-a272-4ac1-a474-a69bb48bbcd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:41.320046 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25a1bb2-a272-4ac1-a474-a69bb48bbcd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:41.320511 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25a1bb2-a272-4ac1-a474-a69bb48bbcd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:41.321558 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25a1bb2-a272-4ac1-a474-a69bb48bbcd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:41.323655 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25a1bb2-a272-4ac1-a474-a69bb48bbcd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:41.324581 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25a1bb2-a272-4ac1-a474-a69bb48bbcd0 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:41.429257 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1117/2236] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:52:41 2022] POST /identity/v3/projects => generated 383 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:52:41.436642 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ff6948-b66b-46ab-84aa-58b74eae9b1c admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:41.453749 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ff6948-b66b-46ab-84aa-58b74eae9b1c admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:41.524978 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff089d6-2aea-4627-ac1d-a71585184586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:41.525649 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff089d6-2aea-4627-ac1d-a71585184586 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:41.525931 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff089d6-2aea-4627-ac1d-a71585184586 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:41.526207 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff089d6-2aea-4627-ac1d-a71585184586 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:41.526901 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff089d6-2aea-4627-ac1d-a71585184586 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:41.528866 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff089d6-2aea-4627-ac1d-a71585184586 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:41.633833 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1120/2237] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:52:41 2022] POST /identity/v3/users => generated 292 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:52:41.638506 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25a1bb2-a272-4ac1-a474-a69bb48bbcd0 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:41.651166 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25a1bb2-a272-4ac1-a474-a69bb48bbcd0 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:41.656315 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b0b6c6-e406-4ff1-8101-cb3da6ccec60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:41.656990 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b0b6c6-e406-4ff1-8101-cb3da6ccec60 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:41.657270 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b0b6c6-e406-4ff1-8101-cb3da6ccec60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:41.657548 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b0b6c6-e406-4ff1-8101-cb3da6ccec60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:41.658274 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b0b6c6-e406-4ff1-8101-cb3da6ccec60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:41.659949 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b0b6c6-e406-4ff1-8101-cb3da6ccec60 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:41.672966 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1118/2238] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:52:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:52:41.679963 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff089d6-2aea-4627-ac1d-a71585184586 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:41.690587 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff089d6-2aea-4627-ac1d-a71585184586 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:41.702572 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6b14cb-3ede-43cc-8f94-cc9a3a079629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:41.705499 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b14cb-3ede-43cc-8f94-cc9a3a079629 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:41.705499 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b14cb-3ede-43cc-8f94-cc9a3a079629 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:41.705499 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b14cb-3ede-43cc-8f94-cc9a3a079629 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:41.705499 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6b14cb-3ede-43cc-8f94-cc9a3a079629 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:41.706417 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6b14cb-3ede-43cc-8f94-cc9a3a079629 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:41.721257 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1121/2239] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:52:41 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:52:41.727534 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b0b6c6-e406-4ff1-8101-cb3da6ccec60 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:41.739519 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b0b6c6-e406-4ff1-8101-cb3da6ccec60 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:41.744962 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288f4236-d547-4935-8a0d-e58da588a57a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:41.745855 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f4236-d547-4935-8a0d-e58da588a57a admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:41.746135 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f4236-d547-4935-8a0d-e58da588a57a admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:41.746412 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f4236-d547-4935-8a0d-e58da588a57a admin admin] PATH_INFO: `/v3/projects/1f578589d6364758b5b358e099b04bf9/users/7f7b9facaad24618a52fa36aea4f3506/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:41.765047 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288f4236-d547-4935-8a0d-e58da588a57a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f578589d6364758b5b358e099b04bf9, user_id=7f7b9facaad24618a52fa36aea4f3506, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:41.767822 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288f4236-d547-4935-8a0d-e58da588a57a admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:41.839762 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1119/2240] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:52:41 2022] PUT /identity/v3/projects/1f578589d6364758b5b358e099b04bf9/users/7f7b9facaad24618a52fa36aea4f3506/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:52:41.850439 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf32823-29d1-4440-9c73-a520e3da5b49 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:41.850723 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf32823-29d1-4440-9c73-a520e3da5b49 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:41.851012 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf32823-29d1-4440-9c73-a520e3da5b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:41.893531 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-fdf32823-29d1-4440-9c73-a520e3da5b49 None None] MFA Rules not processed for user `7f7b9facaad24618a52fa36aea4f3506`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:41.893531 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-fdf32823-29d1-4440-9c73-a520e3da5b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:42.008631 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1122/2241] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:52:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:42.013696 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288f4236-d547-4935-8a0d-e58da588a57a admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:42.084204 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288f4236-d547-4935-8a0d-e58da588a57a admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:42.092365 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca04a5c-3b16-445e-9d26-328e9b5bb141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:42.095936 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca04a5c-3b16-445e-9d26-328e9b5bb141 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:42.095936 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca04a5c-3b16-445e-9d26-328e9b5bb141 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:42.095936 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca04a5c-3b16-445e-9d26-328e9b5bb141 admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:42.095936 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca04a5c-3b16-445e-9d26-328e9b5bb141 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:42.096499 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca04a5c-3b16-445e-9d26-328e9b5bb141 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:42.220682 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1120/2242] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:52:42 2022] POST /identity/v3/projects => generated 383 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:52:42.224947 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf32823-29d1-4440-9c73-a520e3da5b49 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:42.231916 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:42.236362 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf32823-29d1-4440-9c73-a520e3da5b49 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:42.303732 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff48aada-d06e-44c6-b8f4-49f2413c75df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:42.304881 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff48aada-d06e-44c6-b8f4-49f2413c75df admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:42.305647 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff48aada-d06e-44c6-b8f4-49f2413c75df admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:42.306187 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff48aada-d06e-44c6-b8f4-49f2413c75df admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:42.307010 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff48aada-d06e-44c6-b8f4-49f2413c75df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:42.309712 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff48aada-d06e-44c6-b8f4-49f2413c75df admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:42.380998 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1123/2243] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:52:42 2022] POST /identity/v3/users => generated 298 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:52:42.386431 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca04a5c-3b16-445e-9d26-328e9b5bb141 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:42.398172 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca04a5c-3b16-445e-9d26-328e9b5bb141 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:42.406231 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5848f9-7c37-49c4-9ffc-b10474ada376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:42.406971 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5848f9-7c37-49c4-9ffc-b10474ada376 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:42.407294 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5848f9-7c37-49c4-9ffc-b10474ada376 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:42.407621 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5848f9-7c37-49c4-9ffc-b10474ada376 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:42.408455 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5848f9-7c37-49c4-9ffc-b10474ada376 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:42.410228 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5848f9-7c37-49c4-9ffc-b10474ada376 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:42.422187 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1121/2244] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:52:42 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:52:42.429001 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff48aada-d06e-44c6-b8f4-49f2413c75df admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:42.443967 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff48aada-d06e-44c6-b8f4-49f2413c75df admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:42.449964 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395e072b-7838-4ceb-8a57-6094caa3f358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:42.450867 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395e072b-7838-4ceb-8a57-6094caa3f358 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:42.451152 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395e072b-7838-4ceb-8a57-6094caa3f358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:42.451432 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395e072b-7838-4ceb-8a57-6094caa3f358 admin admin] PATH_INFO: `/v3/projects/add1a7e94b20407ca0554b1b32757fcc/users/5542a0a1dfd049e08224981d06e7d1be/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:42.469114 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395e072b-7838-4ceb-8a57-6094caa3f358 admin admin] RBAC: Authorizing `identity:create_grant(project_id=add1a7e94b20407ca0554b1b32757fcc, user_id=5542a0a1dfd049e08224981d06e7d1be, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:42.471861 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395e072b-7838-4ceb-8a57-6094caa3f358 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:42.536682 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1124/2245] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:52:42 2022] PUT /identity/v3/projects/add1a7e94b20407ca0554b1b32757fcc/users/5542a0a1dfd049e08224981d06e7d1be/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:42.541392 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5848f9-7c37-49c4-9ffc-b10474ada376 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:42.568186 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5848f9-7c37-49c4-9ffc-b10474ada376 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:42.637653 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:42.673174 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c492ab8-c687-449b-844b-8bca186f4c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:42.674348 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c492ab8-c687-449b-844b-8bca186f4c51 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:42.674845 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c492ab8-c687-449b-844b-8bca186f4c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:42.675729 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c492ab8-c687-449b-844b-8bca186f4c51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:42.677105 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c492ab8-c687-449b-844b-8bca186f4c51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:42.680756 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c492ab8-c687-449b-844b-8bca186f4c51 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:42.701969 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1122/2246] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:52:42 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:52:42.706531 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395e072b-7838-4ceb-8a57-6094caa3f358 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:42.721885 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395e072b-7838-4ceb-8a57-6094caa3f358 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:42.726852 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ad8f9a-734d-4ceb-bb70-af9188cc06f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:42.727753 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ad8f9a-734d-4ceb-bb70-af9188cc06f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:42.728029 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ad8f9a-734d-4ceb-bb70-af9188cc06f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:42.728348 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ad8f9a-734d-4ceb-bb70-af9188cc06f1 admin admin] PATH_INFO: `/v3/projects/add1a7e94b20407ca0554b1b32757fcc/users/5542a0a1dfd049e08224981d06e7d1be/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:42.730370 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ad8f9a-734d-4ceb-bb70-af9188cc06f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=add1a7e94b20407ca0554b1b32757fcc, user_id=5542a0a1dfd049e08224981d06e7d1be, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:42.733041 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ad8f9a-734d-4ceb-bb70-af9188cc06f1 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:42.744596 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1125/2247] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:52:42 2022] PUT /identity/v3/projects/add1a7e94b20407ca0554b1b32757fcc/users/5542a0a1dfd049e08224981d06e7d1be/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:52:42.756034 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e4b55c-fd7f-46ed-89a9-8ac022b8e090 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:42.756538 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e4b55c-fd7f-46ed-89a9-8ac022b8e090 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:42.756875 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e4b55c-fd7f-46ed-89a9-8ac022b8e090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:42.795570 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-97e4b55c-fd7f-46ed-89a9-8ac022b8e090 None None] MFA Rules not processed for user `5542a0a1dfd049e08224981d06e7d1be`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:42.796835 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-97e4b55c-fd7f-46ed-89a9-8ac022b8e090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:42.871488 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1123/2248] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:52:42 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:52:42.884929 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ad8f9a-734d-4ceb-bb70-af9188cc06f1 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:42.887037 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c1ad8f9a-734d-4ceb-bb70-af9188cc06f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:42.951639 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ad8f9a-734d-4ceb-bb70-af9188cc06f1 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:42.955567 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6f3d1c-a6e6-4fde-985e-6d74cef60b24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:42.956268 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6f3d1c-a6e6-4fde-985e-6d74cef60b24 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:42.956554 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6f3d1c-a6e6-4fde-985e-6d74cef60b24 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:42.956861 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6f3d1c-a6e6-4fde-985e-6d74cef60b24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:42.966946 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6f3d1c-a6e6-4fde-985e-6d74cef60b24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:42.968449 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6f3d1c-a6e6-4fde-985e-6d74cef60b24 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:42.979726 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1126/2249] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:52:42 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:52:43.157923 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e4b55c-fd7f-46ed-89a9-8ac022b8e090 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:43.170653 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e4b55c-fd7f-46ed-89a9-8ac022b8e090 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:43.174817 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ffc555-8c20-424c-b2a2-3c745a1592f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:43.175535 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ffc555-8c20-424c-b2a2-3c745a1592f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:43.175931 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ffc555-8c20-424c-b2a2-3c745a1592f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:43.176735 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ffc555-8c20-424c-b2a2-3c745a1592f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:43.262014 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ffc555-8c20-424c-b2a2-3c745a1592f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:43.264574 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ffc555-8c20-424c-b2a2-3c745a1592f6 service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:43.280572 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1124/2250] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:52:43 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:52:43.874609 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6f3d1c-a6e6-4fde-985e-6d74cef60b24 service cinder] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:43.946464 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6f3d1c-a6e6-4fde-985e-6d74cef60b24 service cinder] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:43.954525 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f8d26-34b0-4eac-bc5d-18fa7ae264ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:43.955395 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f8d26-34b0-4eac-bc5d-18fa7ae264ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:43.955673 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f8d26-34b0-4eac-bc5d-18fa7ae264ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:43.956016 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f8d26-34b0-4eac-bc5d-18fa7ae264ee admin admin] PATH_INFO: `/v3/users/7f7b9facaad24618a52fa36aea4f3506` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:43.957360 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90f8d26-34b0-4eac-bc5d-18fa7ae264ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f7b9facaad24618a52fa36aea4f3506)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:43.959268 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90f8d26-34b0-4eac-bc5d-18fa7ae264ee admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:43.961121 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:44.134881 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-b90f8d26-34b0-4eac-bc5d-18fa7ae264ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f7b9facaad24618a52fa36aea4f3506'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:44.135602 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-b90f8d26-34b0-4eac-bc5d-18fa7ae264ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f7b9facaad24618a52fa36aea4f3506'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:44.136493 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-b90f8d26-34b0-4eac-bc5d-18fa7ae264ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f7b9facaad24618a52fa36aea4f3506'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:44.213097 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-b90f8d26-34b0-4eac-bc5d-18fa7ae264ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f7b9facaad24618a52fa36aea4f3506'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:44.234666 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-b90f8d26-34b0-4eac-bc5d-18fa7ae264ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f7b9facaad24618a52fa36aea4f3506'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:44.251761 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1127/2251] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:52:43 2022] DELETE /identity/v3/users/7f7b9facaad24618a52fa36aea4f3506 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:52:44.256769 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ca4328-ab59-4720-ae19-d4de14f6a402 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:44.257255 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ca4328-ab59-4720-ae19-d4de14f6a402 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:44.257333 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ca4328-ab59-4720-ae19-d4de14f6a402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:44.276412 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-e5ca4328-ab59-4720-ae19-d4de14f6a402 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:44.277635 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-e5ca4328-ab59-4720-ae19-d4de14f6a402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:44.359151 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:44.375586 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1125/2252] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:52:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:52:44.392987 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f8d26-34b0-4eac-bc5d-18fa7ae264ee admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:44.394469 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-b90f8d26-34b0-4eac-bc5d-18fa7ae264ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:44.460482 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f8d26-34b0-4eac-bc5d-18fa7ae264ee admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:44.464180 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c95ab3-15c8-4c3b-a6c7-729a7a02db36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:44.464913 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c95ab3-15c8-4c3b-a6c7-729a7a02db36 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:44.465191 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c95ab3-15c8-4c3b-a6c7-729a7a02db36 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:44.470595 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c95ab3-15c8-4c3b-a6c7-729a7a02db36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:44.476460 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c95ab3-15c8-4c3b-a6c7-729a7a02db36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:44.478714 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c95ab3-15c8-4c3b-a6c7-729a7a02db36 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:44.489548 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1128/2253] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:52:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:52:44.982403 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ca4328-ab59-4720-ae19-d4de14f6a402 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:44.984200 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-e5ca4328-ab59-4720-ae19-d4de14f6a402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:45.007676 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ca4328-ab59-4720-ae19-d4de14f6a402 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:45.017477 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf787d7-bd0a-4d6c-8373-f29ec9c8c846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:45.018320 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf787d7-bd0a-4d6c-8373-f29ec9c8c846 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:45.018690 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf787d7-bd0a-4d6c-8373-f29ec9c8c846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:45.019121 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf787d7-bd0a-4d6c-8373-f29ec9c8c846 admin admin] PATH_INFO: `/v3/projects/1f578589d6364758b5b358e099b04bf9` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:45.020718 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf787d7-bd0a-4d6c-8373-f29ec9c8c846 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f578589d6364758b5b358e099b04bf9)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:45.024081 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf787d7-bd0a-4d6c-8373-f29ec9c8c846 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:45.168299 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-ebf787d7-bd0a-4d6c-8373-f29ec9c8c846 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f578589d6364758b5b358e099b04bf9', 'request_id': 'req-ebf787d7-bd0a-4d6c-8373-f29ec9c8c846'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:45.231785 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-ebf787d7-bd0a-4d6c-8373-f29ec9c8c846 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f578589d6364758b5b358e099b04bf9', 'request_id': 'req-ebf787d7-bd0a-4d6c-8373-f29ec9c8c846'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:45.238357 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-ebf787d7-bd0a-4d6c-8373-f29ec9c8c846 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f578589d6364758b5b358e099b04bf9', 'request_id': 'req-ebf787d7-bd0a-4d6c-8373-f29ec9c8c846'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:45.252059 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-ebf787d7-bd0a-4d6c-8373-f29ec9c8c846 admin admin] The token cache is being invalidate because project 1f578589d6364758b5b358e099b04bf9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:52:45.252386 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-ebf787d7-bd0a-4d6c-8373-f29ec9c8c846 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:45.253105 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-ebf787d7-bd0a-4d6c-8373-f29ec9c8c846 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:45.254686 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1126/2254] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:52:44 2022] DELETE /identity/v3/projects/1f578589d6364758b5b358e099b04bf9 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:45.259581 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c95ab3-15c8-4c3b-a6c7-729a7a02db36 service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:45.270967 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c95ab3-15c8-4c3b-a6c7-729a7a02db36 service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:45.332475 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d695072-e7f6-4e97-b694-d9c869965dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:45.333481 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d695072-e7f6-4e97-b694-d9c869965dd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:45.333849 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d695072-e7f6-4e97-b694-d9c869965dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:45.334173 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d695072-e7f6-4e97-b694-d9c869965dd2 admin admin] PATH_INFO: `/v3/users/5542a0a1dfd049e08224981d06e7d1be` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:45.335541 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d695072-e7f6-4e97-b694-d9c869965dd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5542a0a1dfd049e08224981d06e7d1be)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:45.337674 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d695072-e7f6-4e97-b694-d9c869965dd2 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:45.362717 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:45.526050 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-3d695072-e7f6-4e97-b694-d9c869965dd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5542a0a1dfd049e08224981d06e7d1be'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:45.526851 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-3d695072-e7f6-4e97-b694-d9c869965dd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5542a0a1dfd049e08224981d06e7d1be'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:45.527564 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-3d695072-e7f6-4e97-b694-d9c869965dd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5542a0a1dfd049e08224981d06e7d1be'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:45.605521 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-3d695072-e7f6-4e97-b694-d9c869965dd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5542a0a1dfd049e08224981d06e7d1be'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:45.635493 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-3d695072-e7f6-4e97-b694-d9c869965dd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5542a0a1dfd049e08224981d06e7d1be'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:45.655644 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1129/2255] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:52:45 2022] DELETE /identity/v3/users/5542a0a1dfd049e08224981d06e7d1be => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:46.183435 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf787d7-bd0a-4d6c-8373-f29ec9c8c846 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:46.185582 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-ebf787d7-bd0a-4d6c-8373-f29ec9c8c846 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:46.210586 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf787d7-bd0a-4d6c-8373-f29ec9c8c846 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:46.316854 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706ad7c0-47b1-4916-ba01-84ec00a69006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:46.317723 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706ad7c0-47b1-4916-ba01-84ec00a69006 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:46.318011 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706ad7c0-47b1-4916-ba01-84ec00a69006 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:46.318288 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706ad7c0-47b1-4916-ba01-84ec00a69006 admin admin] PATH_INFO: `/v3/projects/add1a7e94b20407ca0554b1b32757fcc` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:46.319613 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706ad7c0-47b1-4916-ba01-84ec00a69006 admin admin] RBAC: Authorizing `identity:delete_project(project_id=add1a7e94b20407ca0554b1b32757fcc)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:46.322168 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706ad7c0-47b1-4916-ba01-84ec00a69006 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:46.512903 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-706ad7c0-47b1-4916-ba01-84ec00a69006 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'add1a7e94b20407ca0554b1b32757fcc', 'request_id': 'req-706ad7c0-47b1-4916-ba01-84ec00a69006'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:46.603533 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-706ad7c0-47b1-4916-ba01-84ec00a69006 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'add1a7e94b20407ca0554b1b32757fcc', 'request_id': 'req-706ad7c0-47b1-4916-ba01-84ec00a69006'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:46.614560 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-706ad7c0-47b1-4916-ba01-84ec00a69006 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'add1a7e94b20407ca0554b1b32757fcc', 'request_id': 'req-706ad7c0-47b1-4916-ba01-84ec00a69006'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:46.628175 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-706ad7c0-47b1-4916-ba01-84ec00a69006 admin admin] The token cache is being invalidate because project add1a7e94b20407ca0554b1b32757fcc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:52:46.628446 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-706ad7c0-47b1-4916-ba01-84ec00a69006 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:46.629113 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-706ad7c0-47b1-4916-ba01-84ec00a69006 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:46.630676 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1127/2256] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:52:46 2022] DELETE /identity/v3/projects/add1a7e94b20407ca0554b1b32757fcc => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:46.636959 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d169c40e-653a-4f5b-a6c2-27e7488b2256 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:46.637513 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d169c40e-653a-4f5b-a6c2-27e7488b2256 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:46.637513 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d169c40e-653a-4f5b-a6c2-27e7488b2256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:46.655978 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-d169c40e-653a-4f5b-a6c2-27e7488b2256 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:46.657303 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-d169c40e-653a-4f5b-a6c2-27e7488b2256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:46.728707 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1130/2257] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:52:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:46.738267 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cec0a5-0ecd-4559-8d63-1c0c28975b62 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:46.738784 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cec0a5-0ecd-4559-8d63-1c0c28975b62 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:46.739231 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cec0a5-0ecd-4559-8d63-1c0c28975b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:46.778893 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-89cec0a5-0ecd-4559-8d63-1c0c28975b62 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:46.780166 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-89cec0a5-0ecd-4559-8d63-1c0c28975b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:46.793731 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1128/2258] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:52:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:46.798276 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d169c40e-653a-4f5b-a6c2-27e7488b2256 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:46.815143 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d169c40e-653a-4f5b-a6c2-27e7488b2256 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:46.821014 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc2220a-5d7c-408e-8936-c8c6131ed9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:46.821460 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc2220a-5d7c-408e-8936-c8c6131ed9ff admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:46.821743 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc2220a-5d7c-408e-8936-c8c6131ed9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:46.822017 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc2220a-5d7c-408e-8936-c8c6131ed9ff admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:46.822747 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc2220a-5d7c-408e-8936-c8c6131ed9ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:52:46.823054 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc2220a-5d7c-408e-8936-c8c6131ed9ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:46.824826 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc2220a-5d7c-408e-8936-c8c6131ed9ff admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:46.856720 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1131/2259] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jun 15 14:52:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:52:46.858857 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cec0a5-0ecd-4559-8d63-1c0c28975b62 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:46.869272 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cec0a5-0ecd-4559-8d63-1c0c28975b62 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:46.875811 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a05d6f-68da-43e0-a97d-18d674c4a40a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:46.876854 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a05d6f-68da-43e0-a97d-18d674c4a40a admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:46.877120 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a05d6f-68da-43e0-a97d-18d674c4a40a admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:46.877397 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a05d6f-68da-43e0-a97d-18d674c4a40a admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:46.878179 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a05d6f-68da-43e0-a97d-18d674c4a40a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:46.880222 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a05d6f-68da-43e0-a97d-18d674c4a40a admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:46.886344 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:46.989605 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1129/2260] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:52:46 2022] POST /identity/v3/projects => generated 371 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:52:46.994499 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc2220a-5d7c-408e-8936-c8c6131ed9ff admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:47.005712 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc2220a-5d7c-408e-8936-c8c6131ed9ff admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:47.086228 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af975ca8-cf25-411b-a16a-9569f1372d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:47.087022 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af975ca8-cf25-411b-a16a-9569f1372d77 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:47.087305 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af975ca8-cf25-411b-a16a-9569f1372d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:47.087585 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af975ca8-cf25-411b-a16a-9569f1372d77 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:47.088331 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af975ca8-cf25-411b-a16a-9569f1372d77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:47.090452 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af975ca8-cf25-411b-a16a-9569f1372d77 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:47.178303 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1132/2261] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:52:46 2022] POST /identity/v3/users => generated 286 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:52:47.181084 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a05d6f-68da-43e0-a97d-18d674c4a40a admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:47.193953 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a05d6f-68da-43e0-a97d-18d674c4a40a admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:47.204009 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccde23f5-ef61-4248-8447-e849675ee9c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:47.205705 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccde23f5-ef61-4248-8447-e849675ee9c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:47.205995 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccde23f5-ef61-4248-8447-e849675ee9c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:47.206271 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccde23f5-ef61-4248-8447-e849675ee9c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:47.207024 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccde23f5-ef61-4248-8447-e849675ee9c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:47.208687 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccde23f5-ef61-4248-8447-e849675ee9c7 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:47.224206 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1130/2262] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:52:47 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:52:47.227267 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af975ca8-cf25-411b-a16a-9569f1372d77 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:47.242056 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af975ca8-cf25-411b-a16a-9569f1372d77 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:47.244335 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7f86fc-2d3e-4822-98a3-3a48959ec2ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:47.245125 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7f86fc-2d3e-4822-98a3-3a48959ec2ae admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:47.245462 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7f86fc-2d3e-4822-98a3-3a48959ec2ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:47.245804 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7f86fc-2d3e-4822-98a3-3a48959ec2ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:47.246681 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7f86fc-2d3e-4822-98a3-3a48959ec2ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:47.248463 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7f86fc-2d3e-4822-98a3-3a48959ec2ae admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:47.260402 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1133/2263] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:52:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:52:47.265495 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccde23f5-ef61-4248-8447-e849675ee9c7 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:47.282863 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccde23f5-ef61-4248-8447-e849675ee9c7 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:47.289524 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e105a3b-4aa7-4790-a07c-a53cd40e1c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:47.291395 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e105a3b-4aa7-4790-a07c-a53cd40e1c2a admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:47.291752 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e105a3b-4aa7-4790-a07c-a53cd40e1c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:47.292182 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e105a3b-4aa7-4790-a07c-a53cd40e1c2a admin admin] PATH_INFO: `/v3/projects/f83130585c654ba6941c265f16e5eae9/users/a2c9ea6873934c87bcdd07d5a6405e70/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:47.311636 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e105a3b-4aa7-4790-a07c-a53cd40e1c2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f83130585c654ba6941c265f16e5eae9, user_id=a2c9ea6873934c87bcdd07d5a6405e70, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:47.314690 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e105a3b-4aa7-4790-a07c-a53cd40e1c2a admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:47.409057 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1131/2264] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:52:47 2022] PUT /identity/v3/projects/f83130585c654ba6941c265f16e5eae9/users/a2c9ea6873934c87bcdd07d5a6405e70/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:47.413024 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd10b2-2b6e-4ec8-9651-2190f8a86ac5 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:47.413471 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd10b2-2b6e-4ec8-9651-2190f8a86ac5 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:47.414534 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd10b2-2b6e-4ec8-9651-2190f8a86ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:47.435875 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:47.443529 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-25dd10b2-2b6e-4ec8-9651-2190f8a86ac5 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:47.444932 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-25dd10b2-2b6e-4ec8-9651-2190f8a86ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:47.523319 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1134/2265] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:52:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:47.537568 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e105a3b-4aa7-4790-a07c-a53cd40e1c2a admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:47.539661 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-4e105a3b-4aa7-4790-a07c-a53cd40e1c2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:47.619751 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e105a3b-4aa7-4790-a07c-a53cd40e1c2a admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:47.623589 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6343d8a-2843-41fd-ab05-034eb2768653 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:47.624366 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6343d8a-2843-41fd-ab05-034eb2768653 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:47.624695 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6343d8a-2843-41fd-ab05-034eb2768653 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:47.625007 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6343d8a-2843-41fd-ab05-034eb2768653 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:47.636790 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6343d8a-2843-41fd-ab05-034eb2768653 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:47.638246 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6343d8a-2843-41fd-ab05-034eb2768653 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:47.650555 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1132/2266] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:52:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:52:49.187685 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e4e33e-2970-40f5-9019-cb84f70dc078 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:49.188940 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e4e33e-2970-40f5-9019-cb84f70dc078 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:49.188940 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e4e33e-2970-40f5-9019-cb84f70dc078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:49.232306 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-b8e4e33e-2970-40f5-9019-cb84f70dc078 None None] MFA Rules not processed for user `a2c9ea6873934c87bcdd07d5a6405e70`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:49.233691 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-b8e4e33e-2970-40f5-9019-cb84f70dc078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:49.364166 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1135/2267] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:52:49 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:49.369104 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6343d8a-2843-41fd-ab05-034eb2768653 service neutron] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:49.387704 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6343d8a-2843-41fd-ab05-034eb2768653 service neutron] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:49.395254 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3db8bb-3da5-4da9-92f6-35a3267a47a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:49.396157 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3db8bb-3da5-4da9-92f6-35a3267a47a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:49.396440 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3db8bb-3da5-4da9-92f6-35a3267a47a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:49.396800 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3db8bb-3da5-4da9-92f6-35a3267a47a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:49.397696 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3db8bb-3da5-4da9-92f6-35a3267a47a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:49.400321 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3db8bb-3da5-4da9-92f6-35a3267a47a1 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:49.514714 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1133/2268] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:52:49 2022] POST /identity/v3/projects => generated 371 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:52:49.518788 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e4e33e-2970-40f5-9019-cb84f70dc078 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:49.532596 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e4e33e-2970-40f5-9019-cb84f70dc078 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:49.620987 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309eca3a-4c2f-4aa7-a5e5-a4d97ab4eda1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:49.621678 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309eca3a-4c2f-4aa7-a5e5-a4d97ab4eda1 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:49.621962 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309eca3a-4c2f-4aa7-a5e5-a4d97ab4eda1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:49.622241 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309eca3a-4c2f-4aa7-a5e5-a4d97ab4eda1 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:49.622960 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309eca3a-4c2f-4aa7-a5e5-a4d97ab4eda1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:49.624962 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309eca3a-4c2f-4aa7-a5e5-a4d97ab4eda1 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:49.785896 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1136/2269] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:52:49 2022] POST /identity/v3/users => generated 292 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:52:49.787572 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3db8bb-3da5-4da9-92f6-35a3267a47a1 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:49.799403 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3db8bb-3da5-4da9-92f6-35a3267a47a1 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:49.804400 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5acc328-b135-49a6-8b19-ea16a9d7bf1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:49.805591 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5acc328-b135-49a6-8b19-ea16a9d7bf1f admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:49.805880 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5acc328-b135-49a6-8b19-ea16a9d7bf1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:49.808499 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5acc328-b135-49a6-8b19-ea16a9d7bf1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:49.808499 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5acc328-b135-49a6-8b19-ea16a9d7bf1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:49.808767 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5acc328-b135-49a6-8b19-ea16a9d7bf1f admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:49.824946 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1134/2270] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:52:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:52:49.826520 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309eca3a-4c2f-4aa7-a5e5-a4d97ab4eda1 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:49.843967 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309eca3a-4c2f-4aa7-a5e5-a4d97ab4eda1 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:49.851947 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da62b0a1-ca27-47af-a189-c2238a4258fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:49.856048 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da62b0a1-ca27-47af-a189-c2238a4258fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:49.856048 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da62b0a1-ca27-47af-a189-c2238a4258fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:49.856048 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da62b0a1-ca27-47af-a189-c2238a4258fc admin admin] PATH_INFO: `/v3/projects/10f6f219518446f780c46b1fe41b062c/users/25e3f6210dca4a3f967312eb7cdc8923/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:49.873968 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da62b0a1-ca27-47af-a189-c2238a4258fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=10f6f219518446f780c46b1fe41b062c, user_id=25e3f6210dca4a3f967312eb7cdc8923, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:49.876796 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da62b0a1-ca27-47af-a189-c2238a4258fc admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:49.941509 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1137/2271] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:52:49 2022] PUT /identity/v3/projects/10f6f219518446f780c46b1fe41b062c/users/25e3f6210dca4a3f967312eb7cdc8923/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:49.945991 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5acc328-b135-49a6-8b19-ea16a9d7bf1f admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:49.964250 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5acc328-b135-49a6-8b19-ea16a9d7bf1f admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:50.031390 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3406fa21-7689-44e1-9335-c5c4b7177191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:50.032202 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3406fa21-7689-44e1-9335-c5c4b7177191 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:50.032553 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3406fa21-7689-44e1-9335-c5c4b7177191 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:50.032829 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3406fa21-7689-44e1-9335-c5c4b7177191 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:50.033565 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3406fa21-7689-44e1-9335-c5c4b7177191 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:50.035192 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3406fa21-7689-44e1-9335-c5c4b7177191 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:50.051702 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1135/2272] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:52:49 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:52:50.052913 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da62b0a1-ca27-47af-a189-c2238a4258fc admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:50.059312 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:50.066081 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da62b0a1-ca27-47af-a189-c2238a4258fc admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:50.073995 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b26df1f-c21d-415c-93d1-5e35b4d0b97b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:50.074893 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b26df1f-c21d-415c-93d1-5e35b4d0b97b admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:50.075176 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b26df1f-c21d-415c-93d1-5e35b4d0b97b admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:50.075452 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b26df1f-c21d-415c-93d1-5e35b4d0b97b admin admin] PATH_INFO: `/v3/projects/10f6f219518446f780c46b1fe41b062c/users/25e3f6210dca4a3f967312eb7cdc8923/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:50.077771 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b26df1f-c21d-415c-93d1-5e35b4d0b97b admin admin] RBAC: Authorizing `identity:create_grant(project_id=10f6f219518446f780c46b1fe41b062c, user_id=25e3f6210dca4a3f967312eb7cdc8923, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:50.080546 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b26df1f-c21d-415c-93d1-5e35b4d0b97b admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:50.098172 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1138/2273] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:52:50 2022] PUT /identity/v3/projects/10f6f219518446f780c46b1fe41b062c/users/25e3f6210dca4a3f967312eb7cdc8923/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:51.511329 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a3654-0fea-4281-8231-9cbb6f1293bd None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:51.511875 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a3654-0fea-4281-8231-9cbb6f1293bd None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:51.512720 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a3654-0fea-4281-8231-9cbb6f1293bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:51.554207 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-693a3654-0fea-4281-8231-9cbb6f1293bd None None] MFA Rules not processed for user `25e3f6210dca4a3f967312eb7cdc8923`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:51.556782 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-693a3654-0fea-4281-8231-9cbb6f1293bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:51.564469 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:51.674363 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1136/2274] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:52:51 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:51.685494 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b26df1f-c21d-415c-93d1-5e35b4d0b97b admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:51.688015 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-2b26df1f-c21d-415c-93d1-5e35b4d0b97b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:51.775814 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b26df1f-c21d-415c-93d1-5e35b4d0b97b admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:51.779978 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd43ffe-7ece-42ef-9596-c94bbfc2d609 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:51.780812 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd43ffe-7ece-42ef-9596-c94bbfc2d609 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:51.781155 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd43ffe-7ece-42ef-9596-c94bbfc2d609 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:51.781518 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd43ffe-7ece-42ef-9596-c94bbfc2d609 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:51.909897 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd43ffe-7ece-42ef-9596-c94bbfc2d609 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:51.911412 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd43ffe-7ece-42ef-9596-c94bbfc2d609 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:51.923280 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1139/2275] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:52:51 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:52:55.191337 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693a3654-0fea-4281-8231-9cbb6f1293bd None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:55.202243 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693a3654-0fea-4281-8231-9cbb6f1293bd None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:55.206142 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c6d994-16f7-490e-aac5-06c903ca353f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:55.206829 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c6d994-16f7-490e-aac5-06c903ca353f service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:55.207153 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c6d994-16f7-490e-aac5-06c903ca353f service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:55.207437 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c6d994-16f7-490e-aac5-06c903ca353f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:55.221682 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c6d994-16f7-490e-aac5-06c903ca353f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:55.222804 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c6d994-16f7-490e-aac5-06c903ca353f service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:55.235842 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1137/2276] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:52:55 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:53:01.918443 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=52,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:05.231766 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=33 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:10.322398 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5672861-a2d3-474f-8ccb-5887d8e40a76 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:10.330083 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5672861-a2d3-474f-8ccb-5887d8e40a76 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:10.330083 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5672861-a2d3-474f-8ccb-5887d8e40a76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:10.340654 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-b5672861-a2d3-474f-8ccb-5887d8e40a76 None None] MFA Rules not processed for user `ee9df934d1cd461abbf9c3cd90a8fdda`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:10.341877 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-b5672861-a2d3-474f-8ccb-5887d8e40a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:10.398914 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1140/2277] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:53:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:53:12.395773 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c6d994-16f7-490e-aac5-06c903ca353f service cinder] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:12.397181 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-53c6d994-16f7-490e-aac5-06c903ca353f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:12.490438 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c6d994-16f7-490e-aac5-06c903ca353f service cinder] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:12.496914 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107a1a6b-0315-4b37-89ec-6649bc8eee84 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bb6671c19464424a6e9cd4519ac93e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:12.497575 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107a1a6b-0315-4b37-89ec-6649bc8eee84 service placement] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:12.497860 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107a1a6b-0315-4b37-89ec-6649bc8eee84 service placement] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:12.498138 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107a1a6b-0315-4b37-89ec-6649bc8eee84 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:12.499923 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-107a1a6b-0315-4b37-89ec-6649bc8eee84 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:12.516160 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107a1a6b-0315-4b37-89ec-6649bc8eee84 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:12.517799 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107a1a6b-0315-4b37-89ec-6649bc8eee84 service placement] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:12.531320 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1138/2278] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:53:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:53:13.292758 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5672861-a2d3-474f-8ccb-5887d8e40a76 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:13.294183 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-b5672861-a2d3-474f-8ccb-5887d8e40a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:13.364819 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5672861-a2d3-474f-8ccb-5887d8e40a76 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:13.370089 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f1db25-a886-4320-80c2-b7c0ef318732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:13.370974 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1db25-a886-4320-80c2-b7c0ef318732 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:13.371259 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1db25-a886-4320-80c2-b7c0ef318732 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:13.371535 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1db25-a886-4320-80c2-b7c0ef318732 admin admin] PATH_INFO: `/v3/users/74a47ed46ec1496fb6b9b27748c300d3` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:13.372760 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f1db25-a886-4320-80c2-b7c0ef318732 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74a47ed46ec1496fb6b9b27748c300d3)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:13.374693 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f1db25-a886-4320-80c2-b7c0ef318732 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:13.642337 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-50f1db25-a886-4320-80c2-b7c0ef318732 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74a47ed46ec1496fb6b9b27748c300d3'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:13.642993 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-50f1db25-a886-4320-80c2-b7c0ef318732 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74a47ed46ec1496fb6b9b27748c300d3'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:13.643603 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-50f1db25-a886-4320-80c2-b7c0ef318732 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74a47ed46ec1496fb6b9b27748c300d3'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:13.876766 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-50f1db25-a886-4320-80c2-b7c0ef318732 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74a47ed46ec1496fb6b9b27748c300d3'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:13.891555 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-50f1db25-a886-4320-80c2-b7c0ef318732 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74a47ed46ec1496fb6b9b27748c300d3'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:13.905910 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1141/2279] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:53:13 2022] DELETE /identity/v3/users/74a47ed46ec1496fb6b9b27748c300d3 => generated 0 bytes in 614 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:53:13.910347 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdfdf33-1c08-49d7-bb1e-e843dbeba1b6 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:13.910868 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdfdf33-1c08-49d7-bb1e-e843dbeba1b6 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:13.910965 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdfdf33-1c08-49d7-bb1e-e843dbeba1b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:13.928719 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-9fdfdf33-1c08-49d7-bb1e-e843dbeba1b6 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:13.930296 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-9fdfdf33-1c08-49d7-bb1e-e843dbeba1b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:13.995698 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1139/2280] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:53:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:53:14.005415 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f1db25-a886-4320-80c2-b7c0ef318732 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:14.007283 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-50f1db25-a886-4320-80c2-b7c0ef318732 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:14.049501 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:14.106851 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f1db25-a886-4320-80c2-b7c0ef318732 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:14.111464 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bf393d-6719-4bab-bc4a-d50ff284d5a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:14.112275 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bf393d-6719-4bab-bc4a-d50ff284d5a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:14.112626 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bf393d-6719-4bab-bc4a-d50ff284d5a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:14.112993 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bf393d-6719-4bab-bc4a-d50ff284d5a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:14.124700 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bf393d-6719-4bab-bc4a-d50ff284d5a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:14.131383 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bf393d-6719-4bab-bc4a-d50ff284d5a6 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:14.141763 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1142/2281] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:53:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:53:14.798004 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdfdf33-1c08-49d7-bb1e-e843dbeba1b6 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:14.799936 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-9fdfdf33-1c08-49d7-bb1e-e843dbeba1b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:14.832619 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdfdf33-1c08-49d7-bb1e-e843dbeba1b6 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:14.838494 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ea01c0-41f9-4caf-b0e1-7d264fbc7a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:14.839437 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ea01c0-41f9-4caf-b0e1-7d264fbc7a23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:14.839796 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ea01c0-41f9-4caf-b0e1-7d264fbc7a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:14.840191 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ea01c0-41f9-4caf-b0e1-7d264fbc7a23 admin admin] PATH_INFO: `/v3/projects/e8581254187e4316a0f0e1a2f2cd1bf1` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:14.841505 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ea01c0-41f9-4caf-b0e1-7d264fbc7a23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8581254187e4316a0f0e1a2f2cd1bf1)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:14.843491 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ea01c0-41f9-4caf-b0e1-7d264fbc7a23 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:14.941061 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-82ea01c0-41f9-4caf-b0e1-7d264fbc7a23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8581254187e4316a0f0e1a2f2cd1bf1', 'request_id': 'req-82ea01c0-41f9-4caf-b0e1-7d264fbc7a23'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:15.037515 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-82ea01c0-41f9-4caf-b0e1-7d264fbc7a23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8581254187e4316a0f0e1a2f2cd1bf1', 'request_id': 'req-82ea01c0-41f9-4caf-b0e1-7d264fbc7a23'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:15.046820 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-82ea01c0-41f9-4caf-b0e1-7d264fbc7a23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8581254187e4316a0f0e1a2f2cd1bf1', 'request_id': 'req-82ea01c0-41f9-4caf-b0e1-7d264fbc7a23'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:15.046820 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:15.057557 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-82ea01c0-41f9-4caf-b0e1-7d264fbc7a23 admin admin] The token cache is being invalidate because project e8581254187e4316a0f0e1a2f2cd1bf1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:53:15.057978 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-82ea01c0-41f9-4caf-b0e1-7d264fbc7a23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:15.058700 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-82ea01c0-41f9-4caf-b0e1-7d264fbc7a23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:15.060288 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1140/2282] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:53:14 2022] DELETE /identity/v3/projects/e8581254187e4316a0f0e1a2f2cd1bf1 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:53:15.065528 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee74939-dec7-42e1-b66c-bc8bfebbec06 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:15.066046 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee74939-dec7-42e1-b66c-bc8bfebbec06 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:15.066128 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee74939-dec7-42e1-b66c-bc8bfebbec06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:15.085215 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-eee74939-dec7-42e1-b66c-bc8bfebbec06 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:15.087058 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-eee74939-dec7-42e1-b66c-bc8bfebbec06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:15.157438 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1143/2283] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:53:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:53:15.166387 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe3b35-0b3e-4d29-be8c-7e1e694dd424 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:15.166729 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe3b35-0b3e-4d29-be8c-7e1e694dd424 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:15.167018 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe3b35-0b3e-4d29-be8c-7e1e694dd424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:15.187571 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-18fe3b35-0b3e-4d29-be8c-7e1e694dd424 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:15.188738 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-18fe3b35-0b3e-4d29-be8c-7e1e694dd424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:15.198557 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1141/2284] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:53:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:53:15.213750 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee74939-dec7-42e1-b66c-bc8bfebbec06 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:15.223911 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee74939-dec7-42e1-b66c-bc8bfebbec06 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:15.229063 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b086557e-de58-45bb-8331-42b0096715e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:15.229795 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b086557e-de58-45bb-8331-42b0096715e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:15.230101 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b086557e-de58-45bb-8331-42b0096715e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:15.230360 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b086557e-de58-45bb-8331-42b0096715e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:15.231219 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b086557e-de58-45bb-8331-42b0096715e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:53:15.231531 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b086557e-de58-45bb-8331-42b0096715e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:15.233176 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b086557e-de58-45bb-8331-42b0096715e9 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:15.250298 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1144/2285] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jun 15 14:53:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:53:15.253709 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fe3b35-0b3e-4d29-be8c-7e1e694dd424 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:15.265513 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fe3b35-0b3e-4d29-be8c-7e1e694dd424 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:15.271969 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f675f2ec-93b5-40bf-83ac-8678742b4fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:15.272811 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f675f2ec-93b5-40bf-83ac-8678742b4fe3 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:15.273261 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f675f2ec-93b5-40bf-83ac-8678742b4fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:15.274302 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f675f2ec-93b5-40bf-83ac-8678742b4fe3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:15.275325 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f675f2ec-93b5-40bf-83ac-8678742b4fe3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:15.277282 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f675f2ec-93b5-40bf-83ac-8678742b4fe3 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:15.432411 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1142/2286] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:53:15 2022] POST /identity/v3/projects => generated 383 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:53:15.438022 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b086557e-de58-45bb-8331-42b0096715e9 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:15.450100 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b086557e-de58-45bb-8331-42b0096715e9 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:15.514307 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c608d97e-9a13-4df7-992b-3a695b88a388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:15.514976 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c608d97e-9a13-4df7-992b-3a695b88a388 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:15.515258 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c608d97e-9a13-4df7-992b-3a695b88a388 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:15.515534 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c608d97e-9a13-4df7-992b-3a695b88a388 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:15.516299 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c608d97e-9a13-4df7-992b-3a695b88a388 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:15.518224 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c608d97e-9a13-4df7-992b-3a695b88a388 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:15.692171 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1145/2287] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:53:15 2022] POST /identity/v3/users => generated 292 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:53:15.696469 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f675f2ec-93b5-40bf-83ac-8678742b4fe3 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:15.706849 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f675f2ec-93b5-40bf-83ac-8678742b4fe3 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:15.711678 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92e2431-e1b3-4de6-8426-021414632ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:15.717071 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e2431-e1b3-4de6-8426-021414632ada admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:15.717071 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e2431-e1b3-4de6-8426-021414632ada admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:15.717071 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e2431-e1b3-4de6-8426-021414632ada admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:15.717071 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92e2431-e1b3-4de6-8426-021414632ada admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:15.717071 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92e2431-e1b3-4de6-8426-021414632ada admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:15.729641 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1143/2288] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:53:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:53:15.731321 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c608d97e-9a13-4df7-992b-3a695b88a388 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:15.743969 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c608d97e-9a13-4df7-992b-3a695b88a388 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:15.750237 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f382b465-a938-4663-aea8-e90837895e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:15.751058 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f382b465-a938-4663-aea8-e90837895e6d admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:15.751441 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f382b465-a938-4663-aea8-e90837895e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:15.751835 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f382b465-a938-4663-aea8-e90837895e6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:15.752761 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f382b465-a938-4663-aea8-e90837895e6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:15.754587 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f382b465-a938-4663-aea8-e90837895e6d admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:15.768573 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1146/2289] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:53:15 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:53:15.772717 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92e2431-e1b3-4de6-8426-021414632ada admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:15.789394 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92e2431-e1b3-4de6-8426-021414632ada admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:15.796314 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9254f2a7-1254-4c07-b398-974683f20464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:15.797445 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9254f2a7-1254-4c07-b398-974683f20464 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:15.797816 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9254f2a7-1254-4c07-b398-974683f20464 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:15.798338 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9254f2a7-1254-4c07-b398-974683f20464 admin admin] PATH_INFO: `/v3/projects/f49269ee4f394df2a7262608060953d6/users/b51b59d6618e409ebb0ef6bc03ef79bd/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:15.838302 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9254f2a7-1254-4c07-b398-974683f20464 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f49269ee4f394df2a7262608060953d6, user_id=b51b59d6618e409ebb0ef6bc03ef79bd, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:15.841398 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9254f2a7-1254-4c07-b398-974683f20464 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:15.979823 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1144/2290] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:53:15 2022] PUT /identity/v3/projects/f49269ee4f394df2a7262608060953d6/users/b51b59d6618e409ebb0ef6bc03ef79bd/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:53:15.992302 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f224b1-6366-44ab-b7a5-546557fa9c97 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:15.992878 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f224b1-6366-44ab-b7a5-546557fa9c97 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:15.993745 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f224b1-6366-44ab-b7a5-546557fa9c97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:16.028081 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-e9f224b1-6366-44ab-b7a5-546557fa9c97 None None] MFA Rules not processed for user `b51b59d6618e409ebb0ef6bc03ef79bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:16.029456 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e9f224b1-6366-44ab-b7a5-546557fa9c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:16.051130 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:16.095560 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1147/2291] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:53:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:53:16.108528 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9254f2a7-1254-4c07-b398-974683f20464 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:16.110379 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-9254f2a7-1254-4c07-b398-974683f20464 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:16.185016 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9254f2a7-1254-4c07-b398-974683f20464 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:16.190023 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21d9f04-e06c-4d4f-970f-c0b2e375705d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:16.190023 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21d9f04-e06c-4d4f-970f-c0b2e375705d service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:16.190566 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21d9f04-e06c-4d4f-970f-c0b2e375705d service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:16.190566 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21d9f04-e06c-4d4f-970f-c0b2e375705d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:16.201485 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21d9f04-e06c-4d4f-970f-c0b2e375705d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:16.203192 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21d9f04-e06c-4d4f-970f-c0b2e375705d service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:16.217351 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1145/2292] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:53:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:53:20.114238 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f224b1-6366-44ab-b7a5-546557fa9c97 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:20.115671 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e9f224b1-6366-44ab-b7a5-546557fa9c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:20.181936 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f224b1-6366-44ab-b7a5-546557fa9c97 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:20.186713 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb09922-8447-44e3-9550-3d0a958f44a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:20.187606 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb09922-8447-44e3-9550-3d0a958f44a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:20.187886 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb09922-8447-44e3-9550-3d0a958f44a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:20.188240 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb09922-8447-44e3-9550-3d0a958f44a9 admin admin] PATH_INFO: `/v3/users/5826763f3c24400d9016394999717e50` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:20.189475 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb09922-8447-44e3-9550-3d0a958f44a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5826763f3c24400d9016394999717e50)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:20.191438 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb09922-8447-44e3-9550-3d0a958f44a9 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:20.411744 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-cdb09922-8447-44e3-9550-3d0a958f44a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5826763f3c24400d9016394999717e50'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:20.412426 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-cdb09922-8447-44e3-9550-3d0a958f44a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5826763f3c24400d9016394999717e50'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:20.413052 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-cdb09922-8447-44e3-9550-3d0a958f44a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5826763f3c24400d9016394999717e50'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:20.474618 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-cdb09922-8447-44e3-9550-3d0a958f44a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5826763f3c24400d9016394999717e50'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:20.488562 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-cdb09922-8447-44e3-9550-3d0a958f44a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5826763f3c24400d9016394999717e50'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:20.501094 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1148/2293] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:53:20 2022] DELETE /identity/v3/users/5826763f3c24400d9016394999717e50 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:53:20.505884 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26079f0f-0e26-4d44-b216-784c5b587567 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:20.506353 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26079f0f-0e26-4d44-b216-784c5b587567 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:20.507375 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26079f0f-0e26-4d44-b216-784c5b587567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:20.527105 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-26079f0f-0e26-4d44-b216-784c5b587567 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:20.528367 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-26079f0f-0e26-4d44-b216-784c5b587567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:20.591195 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1146/2294] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:53:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:53:20.600884 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb09922-8447-44e3-9550-3d0a958f44a9 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:20.602332 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-cdb09922-8447-44e3-9550-3d0a958f44a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:20.651576 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:20.663156 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb09922-8447-44e3-9550-3d0a958f44a9 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:20.667494 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc474ce-cf01-44af-9888-f79ee1650874 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:20.668244 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc474ce-cf01-44af-9888-f79ee1650874 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:20.668583 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc474ce-cf01-44af-9888-f79ee1650874 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:20.668859 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc474ce-cf01-44af-9888-f79ee1650874 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:20.678776 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc474ce-cf01-44af-9888-f79ee1650874 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:20.680393 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc474ce-cf01-44af-9888-f79ee1650874 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:20.690420 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1149/2295] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:53:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:53:21.100427 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26079f0f-0e26-4d44-b216-784c5b587567 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:21.101857 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-26079f0f-0e26-4d44-b216-784c5b587567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:21.114322 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26079f0f-0e26-4d44-b216-784c5b587567 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:21.120593 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3e5218-9e97-4ef6-b740-bd1bf968fad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:21.121874 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e5218-9e97-4ef6-b740-bd1bf968fad7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:21.122165 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e5218-9e97-4ef6-b740-bd1bf968fad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:21.122442 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e5218-9e97-4ef6-b740-bd1bf968fad7 admin admin] PATH_INFO: `/v3/projects/f5511d18ecfb4e32a6398261be03a447` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:21.123707 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3e5218-9e97-4ef6-b740-bd1bf968fad7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5511d18ecfb4e32a6398261be03a447)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:21.130504 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3e5218-9e97-4ef6-b740-bd1bf968fad7 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:21.239832 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:21.255532 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-6c3e5218-9e97-4ef6-b740-bd1bf968fad7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5511d18ecfb4e32a6398261be03a447', 'request_id': 'req-6c3e5218-9e97-4ef6-b740-bd1bf968fad7'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:21.327341 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-6c3e5218-9e97-4ef6-b740-bd1bf968fad7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5511d18ecfb4e32a6398261be03a447', 'request_id': 'req-6c3e5218-9e97-4ef6-b740-bd1bf968fad7'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:21.333664 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-6c3e5218-9e97-4ef6-b740-bd1bf968fad7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5511d18ecfb4e32a6398261be03a447', 'request_id': 'req-6c3e5218-9e97-4ef6-b740-bd1bf968fad7'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:21.345140 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-6c3e5218-9e97-4ef6-b740-bd1bf968fad7 admin admin] The token cache is being invalidate because project f5511d18ecfb4e32a6398261be03a447 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:53:21.345430 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-6c3e5218-9e97-4ef6-b740-bd1bf968fad7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:21.346097 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-6c3e5218-9e97-4ef6-b740-bd1bf968fad7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:21.347643 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1147/2296] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:53:21 2022] DELETE /identity/v3/projects/f5511d18ecfb4e32a6398261be03a447 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:53:21.352570 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9206c3-d5ee-4602-8cfe-7536dc3e18fc None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:21.353089 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9206c3-d5ee-4602-8cfe-7536dc3e18fc None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:21.353184 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9206c3-d5ee-4602-8cfe-7536dc3e18fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:21.372265 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-3a9206c3-d5ee-4602-8cfe-7536dc3e18fc None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:21.373420 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-3a9206c3-d5ee-4602-8cfe-7536dc3e18fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:21.438774 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1150/2297] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:53:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:53:21.447560 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238512da-4b62-4f88-9ae1-e5150cc039b2 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:21.448128 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238512da-4b62-4f88-9ae1-e5150cc039b2 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:21.448598 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238512da-4b62-4f88-9ae1-e5150cc039b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:21.469824 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-238512da-4b62-4f88-9ae1-e5150cc039b2 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:21.471718 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-238512da-4b62-4f88-9ae1-e5150cc039b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:21.480990 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1148/2298] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:53:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:53:21.485091 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9206c3-d5ee-4602-8cfe-7536dc3e18fc None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:21.495992 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9206c3-d5ee-4602-8cfe-7536dc3e18fc None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:21.500920 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec0253c-95d7-420d-bdd4-41cc8efd5e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:21.501581 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0253c-95d7-420d-bdd4-41cc8efd5e57 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:21.501859 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0253c-95d7-420d-bdd4-41cc8efd5e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:21.502134 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0253c-95d7-420d-bdd4-41cc8efd5e57 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:21.503515 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec0253c-95d7-420d-bdd4-41cc8efd5e57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:53:21.503830 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec0253c-95d7-420d-bdd4-41cc8efd5e57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:21.505462 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec0253c-95d7-420d-bdd4-41cc8efd5e57 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:21.518493 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1151/2299] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jun 15 14:53:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:53:21.522838 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238512da-4b62-4f88-9ae1-e5150cc039b2 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:21.533708 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238512da-4b62-4f88-9ae1-e5150cc039b2 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:21.538640 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9c854b-da50-4fc8-b842-0b2842264a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:21.539317 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9c854b-da50-4fc8-b842-0b2842264a1c admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:21.539746 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9c854b-da50-4fc8-b842-0b2842264a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:21.540028 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9c854b-da50-4fc8-b842-0b2842264a1c admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:21.542368 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9c854b-da50-4fc8-b842-0b2842264a1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:21.543806 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9c854b-da50-4fc8-b842-0b2842264a1c admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:21.653379 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1149/2300] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:53:21 2022] POST /identity/v3/projects => generated 381 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:53:21.657676 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec0253c-95d7-420d-bdd4-41cc8efd5e57 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:21.668869 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec0253c-95d7-420d-bdd4-41cc8efd5e57 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:21.727896 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441a25b4-23e2-4e11-9d88-4a80645d4224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:21.728758 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441a25b4-23e2-4e11-9d88-4a80645d4224 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:21.729043 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441a25b4-23e2-4e11-9d88-4a80645d4224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:21.729319 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441a25b4-23e2-4e11-9d88-4a80645d4224 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:21.730068 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441a25b4-23e2-4e11-9d88-4a80645d4224 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:21.731993 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441a25b4-23e2-4e11-9d88-4a80645d4224 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:21.778672 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9c854b-da50-4fc8-b842-0b2842264a1c admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:21.780548 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-cd9c854b-da50-4fc8-b842-0b2842264a1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:21.794695 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9c854b-da50-4fc8-b842-0b2842264a1c admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:21.799885 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54769351-6e83-4569-ba59-4cc9015e070e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:21.800872 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54769351-6e83-4569-ba59-4cc9015e070e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:21.801365 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54769351-6e83-4569-ba59-4cc9015e070e admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:21.801725 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54769351-6e83-4569-ba59-4cc9015e070e admin admin] PATH_INFO: `/v3/users/b51b59d6618e409ebb0ef6bc03ef79bd` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:21.803081 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54769351-6e83-4569-ba59-4cc9015e070e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b51b59d6618e409ebb0ef6bc03ef79bd)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:21.805395 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54769351-6e83-4569-ba59-4cc9015e070e admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:21.829369 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1152/2301] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:53:21 2022] POST /identity/v3/users => generated 291 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:53:21.832969 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441a25b4-23e2-4e11-9d88-4a80645d4224 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:21.848421 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441a25b4-23e2-4e11-9d88-4a80645d4224 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:21.853933 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f193cf0b-5791-44fc-9139-60281c77316f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:21.854695 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f193cf0b-5791-44fc-9139-60281c77316f admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:21.855047 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f193cf0b-5791-44fc-9139-60281c77316f admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:21.855395 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f193cf0b-5791-44fc-9139-60281c77316f admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:21.856416 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f193cf0b-5791-44fc-9139-60281c77316f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:21.858074 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f193cf0b-5791-44fc-9139-60281c77316f admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:21.869373 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1153/2302] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:53:21 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:53:21.876715 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f193cf0b-5791-44fc-9139-60281c77316f admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:21.883402 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f193cf0b-5791-44fc-9139-60281c77316f admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:21.889110 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4d92b2-3a66-4df2-825b-4dd16ca077cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:21.889778 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d92b2-3a66-4df2-825b-4dd16ca077cc admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:21.890058 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d92b2-3a66-4df2-825b-4dd16ca077cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:21.890339 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d92b2-3a66-4df2-825b-4dd16ca077cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:21.891072 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4d92b2-3a66-4df2-825b-4dd16ca077cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:21.892721 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4d92b2-3a66-4df2-825b-4dd16ca077cc admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:21.904531 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1154/2303] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:53:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:53:21.908629 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4d92b2-3a66-4df2-825b-4dd16ca077cc admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:21.919118 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4d92b2-3a66-4df2-825b-4dd16ca077cc admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:21.923963 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322e2078-9205-4de6-b931-0fece6cfe915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:21.924877 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322e2078-9205-4de6-b931-0fece6cfe915 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:21.925153 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322e2078-9205-4de6-b931-0fece6cfe915 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:21.925422 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322e2078-9205-4de6-b931-0fece6cfe915 admin admin] PATH_INFO: `/v3/projects/b2cdfc40cdd4460f8d6bae576e144ec2/users/66cc9f37bc2d4634875587ed1b2bf774/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:21.942447 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322e2078-9205-4de6-b931-0fece6cfe915 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2cdfc40cdd4460f8d6bae576e144ec2, user_id=66cc9f37bc2d4634875587ed1b2bf774, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:21.945261 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322e2078-9205-4de6-b931-0fece6cfe915 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:21.974373 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-54769351-6e83-4569-ba59-4cc9015e070e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b51b59d6618e409ebb0ef6bc03ef79bd'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:21.975107 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-54769351-6e83-4569-ba59-4cc9015e070e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b51b59d6618e409ebb0ef6bc03ef79bd'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:21.982304 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-54769351-6e83-4569-ba59-4cc9015e070e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b51b59d6618e409ebb0ef6bc03ef79bd'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:21.983001 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-54769351-6e83-4569-ba59-4cc9015e070e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b51b59d6618e409ebb0ef6bc03ef79bd'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:22.021923 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1155/2304] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:53:21 2022] PUT /identity/v3/projects/b2cdfc40cdd4460f8d6bae576e144ec2/users/66cc9f37bc2d4634875587ed1b2bf774/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:53:22.034030 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcefcf88-8346-4092-b094-036fa9a99363 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:22.034434 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcefcf88-8346-4092-b094-036fa9a99363 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:22.034775 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcefcf88-8346-4092-b094-036fa9a99363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:22.063288 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-bcefcf88-8346-4092-b094-036fa9a99363 None None] MFA Rules not processed for user `66cc9f37bc2d4634875587ed1b2bf774`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:22.064548 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-bcefcf88-8346-4092-b094-036fa9a99363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:22.065098 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-54769351-6e83-4569-ba59-4cc9015e070e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b51b59d6618e409ebb0ef6bc03ef79bd'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:22.069385 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:22.086402 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1150/2305] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:53:21 2022] DELETE /identity/v3/users/b51b59d6618e409ebb0ef6bc03ef79bd => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:53:22.091465 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81603a2d-a5f0-4aaa-a198-f1faea2c4bac None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:22.091751 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81603a2d-a5f0-4aaa-a198-f1faea2c4bac None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:22.092027 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81603a2d-a5f0-4aaa-a198-f1faea2c4bac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:22.109956 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-81603a2d-a5f0-4aaa-a198-f1faea2c4bac None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:22.111440 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-81603a2d-a5f0-4aaa-a198-f1faea2c4bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:22.139681 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1156/2306] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:53:22 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:53:22.141004 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcefcf88-8346-4092-b094-036fa9a99363 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:22.142722 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-bcefcf88-8346-4092-b094-036fa9a99363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:22.175532 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=82,keystone:DELETE=15,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:22.186225 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1151/2307] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:53:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:53:22.198063 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81603a2d-a5f0-4aaa-a198-f1faea2c4bac None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:22.199490 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-81603a2d-a5f0-4aaa-a198-f1faea2c4bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:22.227022 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcefcf88-8346-4092-b094-036fa9a99363 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:22.232523 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbd0b7c-a8b2-43de-92ff-fd107675b402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:22.234097 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbd0b7c-a8b2-43de-92ff-fd107675b402 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:22.234532 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbd0b7c-a8b2-43de-92ff-fd107675b402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:22.235009 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbd0b7c-a8b2-43de-92ff-fd107675b402 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:22.235683 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbd0b7c-a8b2-43de-92ff-fd107675b402 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:22.237679 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbd0b7c-a8b2-43de-92ff-fd107675b402 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:22.266009 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81603a2d-a5f0-4aaa-a198-f1faea2c4bac None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:22.269733 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29d691-7ce8-44e7-933a-58933391593f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:22.270390 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29d691-7ce8-44e7-933a-58933391593f service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:22.270672 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29d691-7ce8-44e7-933a-58933391593f service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:22.270950 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29d691-7ce8-44e7-933a-58933391593f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:22.281647 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb29d691-7ce8-44e7-933a-58933391593f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:22.283123 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb29d691-7ce8-44e7-933a-58933391593f service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:22.293428 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1152/2308] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:53:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:53:22.342424 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1157/2309] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:53:22 2022] POST /identity/v3/projects => generated 385 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:53:22.346360 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29d691-7ce8-44e7-933a-58933391593f service neutron] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:22.356875 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29d691-7ce8-44e7-933a-58933391593f service neutron] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:22.419202 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef96bfe8-9e88-412d-a08c-8fec284a1b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:22.419797 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef96bfe8-9e88-412d-a08c-8fec284a1b8b admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:22.420130 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef96bfe8-9e88-412d-a08c-8fec284a1b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:22.420416 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef96bfe8-9e88-412d-a08c-8fec284a1b8b admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:22.421264 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef96bfe8-9e88-412d-a08c-8fec284a1b8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:22.423237 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef96bfe8-9e88-412d-a08c-8fec284a1b8b admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:22.602621 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1153/2310] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:53:22 2022] POST /identity/v3/users => generated 293 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:53:22.607177 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbd0b7c-a8b2-43de-92ff-fd107675b402 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:22.617609 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbd0b7c-a8b2-43de-92ff-fd107675b402 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:22.622960 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f3a7eb-6f00-4096-b43a-f7abacf68805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:22.623751 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f3a7eb-6f00-4096-b43a-f7abacf68805 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:22.624230 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f3a7eb-6f00-4096-b43a-f7abacf68805 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:22.624528 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f3a7eb-6f00-4096-b43a-f7abacf68805 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:22.625331 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f3a7eb-6f00-4096-b43a-f7abacf68805 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:22.626928 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f3a7eb-6f00-4096-b43a-f7abacf68805 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:22.640704 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1158/2311] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:53:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:53:22.644768 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef96bfe8-9e88-412d-a08c-8fec284a1b8b admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:22.655775 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef96bfe8-9e88-412d-a08c-8fec284a1b8b admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:22.661411 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c26cc5-5c27-4123-bbe3-f79638ae4c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:22.662102 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c26cc5-5c27-4123-bbe3-f79638ae4c99 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:22.662661 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c26cc5-5c27-4123-bbe3-f79638ae4c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:22.662950 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c26cc5-5c27-4123-bbe3-f79638ae4c99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:22.663715 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c26cc5-5c27-4123-bbe3-f79638ae4c99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:22.665871 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c26cc5-5c27-4123-bbe3-f79638ae4c99 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:22.682077 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f3a7eb-6f00-4096-b43a-f7abacf68805 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:22.684250 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1154/2312] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:53:22 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:53:22.693415 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f3a7eb-6f00-4096-b43a-f7abacf68805 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:22.699600 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b66b8e-19ce-43c4-bdad-5eb35112fada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:22.700536 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b66b8e-19ce-43c4-bdad-5eb35112fada admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:22.701073 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b66b8e-19ce-43c4-bdad-5eb35112fada admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:22.701343 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b66b8e-19ce-43c4-bdad-5eb35112fada admin admin] PATH_INFO: `/v3/projects/c50914d0345048fab34fd0ccfb91e190/users/1849b699f701423b91e98f3dad25ca96/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:22.720228 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b66b8e-19ce-43c4-bdad-5eb35112fada admin admin] RBAC: Authorizing `identity:create_grant(project_id=c50914d0345048fab34fd0ccfb91e190, user_id=1849b699f701423b91e98f3dad25ca96, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:22.722927 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b66b8e-19ce-43c4-bdad-5eb35112fada admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:22.866498 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c26cc5-5c27-4123-bbe3-f79638ae4c99 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:22.869253 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-58c26cc5-5c27-4123-bbe3-f79638ae4c99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:22.886823 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c26cc5-5c27-4123-bbe3-f79638ae4c99 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:22.893209 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf0e559-0826-41a4-9d78-37ffe24d5864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:22.893894 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf0e559-0826-41a4-9d78-37ffe24d5864 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:22.894176 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf0e559-0826-41a4-9d78-37ffe24d5864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:22.894450 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf0e559-0826-41a4-9d78-37ffe24d5864 admin admin] PATH_INFO: `/v3/projects/f49269ee4f394df2a7262608060953d6` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:22.895751 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf0e559-0826-41a4-9d78-37ffe24d5864 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f49269ee4f394df2a7262608060953d6)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:22.897907 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf0e559-0826-41a4-9d78-37ffe24d5864 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:22.974011 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1159/2313] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:53:22 2022] PUT /identity/v3/projects/c50914d0345048fab34fd0ccfb91e190/users/1849b699f701423b91e98f3dad25ca96/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:53:22.986264 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e376d6-7596-4186-9728-6924b8163bd0 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:22.986585 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e376d6-7596-4186-9728-6924b8163bd0 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:22.986837 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e376d6-7596-4186-9728-6924b8163bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:23.016423 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-21e376d6-7596-4186-9728-6924b8163bd0 None None] MFA Rules not processed for user `1849b699f701423b91e98f3dad25ca96`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:23.214753 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-21e376d6-7596-4186-9728-6924b8163bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:23.214753 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1160/2314] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:53:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:53:23.214753 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e376d6-7596-4186-9728-6924b8163bd0 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:23.277673 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e376d6-7596-4186-9728-6924b8163bd0 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:23.283468 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d427e-079f-4088-830e-b5e71239f4a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:23.284297 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d427e-079f-4088-830e-b5e71239f4a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:23.284621 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d427e-079f-4088-830e-b5e71239f4a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:23.285072 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d427e-079f-4088-830e-b5e71239f4a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:23.285683 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8d427e-079f-4088-830e-b5e71239f4a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:23.287885 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8d427e-079f-4088-830e-b5e71239f4a9 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:23.315524 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-4cf0e559-0826-41a4-9d78-37ffe24d5864 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f49269ee4f394df2a7262608060953d6', 'request_id': 'req-4cf0e559-0826-41a4-9d78-37ffe24d5864'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:23.602564 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-4cf0e559-0826-41a4-9d78-37ffe24d5864 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f49269ee4f394df2a7262608060953d6', 'request_id': 'req-4cf0e559-0826-41a4-9d78-37ffe24d5864'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:23.612616 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-4cf0e559-0826-41a4-9d78-37ffe24d5864 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f49269ee4f394df2a7262608060953d6', 'request_id': 'req-4cf0e559-0826-41a4-9d78-37ffe24d5864'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:23.614238 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1161/2315] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:53:23 2022] POST /identity/v3/projects => generated 385 bytes in 422 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:53:23.615107 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d427e-079f-4088-830e-b5e71239f4a9 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:23.630472 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-4cf0e559-0826-41a4-9d78-37ffe24d5864 admin admin] The token cache is being invalidate because project f49269ee4f394df2a7262608060953d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:53:23.630832 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-4cf0e559-0826-41a4-9d78-37ffe24d5864 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:23.631581 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-4cf0e559-0826-41a4-9d78-37ffe24d5864 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:23.633569 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1155/2316] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:53:22 2022] DELETE /identity/v3/projects/f49269ee4f394df2a7262608060953d6 => generated 0 bytes in 768 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:53:23.640572 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e54530-3afe-4201-b80f-1f0cad04bae4 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:23.640859 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e54530-3afe-4201-b80f-1f0cad04bae4 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:23.641142 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e54530-3afe-4201-b80f-1f0cad04bae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:23.664602 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-64e54530-3afe-4201-b80f-1f0cad04bae4 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:23.667154 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-64e54530-3afe-4201-b80f-1f0cad04bae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:23.711839 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d427e-079f-4088-830e-b5e71239f4a9 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:23.714433 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be17932b-6913-4fe6-ac71-9101c3b82b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:23.715134 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17932b-6913-4fe6-ac71-9101c3b82b80 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:23.715422 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17932b-6913-4fe6-ac71-9101c3b82b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:23.715732 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17932b-6913-4fe6-ac71-9101c3b82b80 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:23.716558 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be17932b-6913-4fe6-ac71-9101c3b82b80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:23.718940 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be17932b-6913-4fe6-ac71-9101c3b82b80 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:23.748542 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1156/2317] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:53:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:53:23.756941 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864ac66-55b9-475b-b6bb-77550a895ecd None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:23.760073 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864ac66-55b9-475b-b6bb-77550a895ecd None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:23.760073 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864ac66-55b9-475b-b6bb-77550a895ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:23.780177 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:23.784672 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-7864ac66-55b9-475b-b6bb-77550a895ecd None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:23.788452 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-7864ac66-55b9-475b-b6bb-77550a895ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:23.802025 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1157/2318] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:53:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:53:23.806472 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7864ac66-55b9-475b-b6bb-77550a895ecd None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:23.815976 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1162/2319] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:53:23 2022] POST /identity/v3/users => generated 299 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:53:23.817118 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7864ac66-55b9-475b-b6bb-77550a895ecd None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:23.823192 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be17932b-6913-4fe6-ac71-9101c3b82b80 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:23.824065 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deba5ad-850c-4c95-9d5a-8ba67912b9e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:23.824774 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deba5ad-850c-4c95-9d5a-8ba67912b9e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:23.825007 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-be17932b-6913-4fe6-ac71-9101c3b82b80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:23.825122 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deba5ad-850c-4c95-9d5a-8ba67912b9e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:23.825321 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deba5ad-850c-4c95-9d5a-8ba67912b9e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:23.826431 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deba5ad-850c-4c95-9d5a-8ba67912b9e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:53:23.826891 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deba5ad-850c-4c95-9d5a-8ba67912b9e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:23.828586 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deba5ad-850c-4c95-9d5a-8ba67912b9e9 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:23.841838 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be17932b-6913-4fe6-ac71-9101c3b82b80 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:23.849791 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1158/2320] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jun 15 14:53:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:53:23.850252 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862c7c8c-c17b-4dc2-897c-f8be2eada606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:23.850937 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862c7c8c-c17b-4dc2-897c-f8be2eada606 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:23.851225 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862c7c8c-c17b-4dc2-897c-f8be2eada606 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:23.851508 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862c7c8c-c17b-4dc2-897c-f8be2eada606 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:23.852275 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862c7c8c-c17b-4dc2-897c-f8be2eada606 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:23.853902 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deba5ad-850c-4c95-9d5a-8ba67912b9e9 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:23.854196 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862c7c8c-c17b-4dc2-897c-f8be2eada606 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:23.866959 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1163/2321] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:53:23 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:53:23.867223 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deba5ad-850c-4c95-9d5a-8ba67912b9e9 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:23.872969 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862c7c8c-c17b-4dc2-897c-f8be2eada606 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:23.873983 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7fbcc1-ae5e-47c6-ab0f-7a35d72164a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:23.874869 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7fbcc1-ae5e-47c6-ab0f-7a35d72164a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:23.875323 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7fbcc1-ae5e-47c6-ab0f-7a35d72164a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:23.875777 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7fbcc1-ae5e-47c6-ab0f-7a35d72164a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:23.878310 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7fbcc1-ae5e-47c6-ab0f-7a35d72164a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:23.881042 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7fbcc1-ae5e-47c6-ab0f-7a35d72164a7 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:23.893211 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862c7c8c-c17b-4dc2-897c-f8be2eada606 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:23.902729 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa036b4-016d-4e50-b3de-80d3df29a215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:23.903790 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa036b4-016d-4e50-b3de-80d3df29a215 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:23.907849 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa036b4-016d-4e50-b3de-80d3df29a215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:23.907849 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa036b4-016d-4e50-b3de-80d3df29a215 admin admin] PATH_INFO: `/v3/projects/cf6fb66889b4489c9247928098504643/users/6fe52fbff8294575bed5a1dbb6cd78a9/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:23.925594 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa036b4-016d-4e50-b3de-80d3df29a215 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf6fb66889b4489c9247928098504643, user_id=6fe52fbff8294575bed5a1dbb6cd78a9, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:23.928657 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa036b4-016d-4e50-b3de-80d3df29a215 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:23.991611 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1159/2322] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:53:23 2022] POST /identity/v3/projects => generated 375 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:53:23.991611 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7fbcc1-ae5e-47c6-ab0f-7a35d72164a7 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:24.032676 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1164/2323] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:53:23 2022] PUT /identity/v3/projects/cf6fb66889b4489c9247928098504643/users/6fe52fbff8294575bed5a1dbb6cd78a9/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:53:24.036977 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa036b4-016d-4e50-b3de-80d3df29a215 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:24.060524 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7fbcc1-ae5e-47c6-ab0f-7a35d72164a7 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:24.129100 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa036b4-016d-4e50-b3de-80d3df29a215 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:24.136200 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b251b4-6cd6-4565-bc33-ee9c7e16541a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:24.137262 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b251b4-6cd6-4565-bc33-ee9c7e16541a admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:24.137694 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b251b4-6cd6-4565-bc33-ee9c7e16541a admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:24.137805 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5eba69-012e-4e9c-82d7-5093c8e7aa22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:24.138144 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b251b4-6cd6-4565-bc33-ee9c7e16541a admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:24.138868 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5eba69-012e-4e9c-82d7-5093c8e7aa22 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:24.139265 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b251b4-6cd6-4565-bc33-ee9c7e16541a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:24.139374 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5eba69-012e-4e9c-82d7-5093c8e7aa22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:24.139715 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5eba69-012e-4e9c-82d7-5093c8e7aa22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:24.140967 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5eba69-012e-4e9c-82d7-5093c8e7aa22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:24.142783 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b251b4-6cd6-4565-bc33-ee9c7e16541a admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:24.143762 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5eba69-012e-4e9c-82d7-5093c8e7aa22 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:24.161387 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1165/2324] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:53:24 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:53:24.166168 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5eba69-012e-4e9c-82d7-5093c8e7aa22 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:24.176307 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5eba69-012e-4e9c-82d7-5093c8e7aa22 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:24.181180 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3088add6-3f66-4c39-858d-93d3e468e87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:24.182192 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3088add6-3f66-4c39-858d-93d3e468e87e admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:24.182471 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3088add6-3f66-4c39-858d-93d3e468e87e admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:24.182752 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3088add6-3f66-4c39-858d-93d3e468e87e admin admin] PATH_INFO: `/v3/projects/cf6fb66889b4489c9247928098504643/users/6fe52fbff8294575bed5a1dbb6cd78a9/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:24.184828 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3088add6-3f66-4c39-858d-93d3e468e87e admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf6fb66889b4489c9247928098504643, user_id=6fe52fbff8294575bed5a1dbb6cd78a9, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:24.187578 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3088add6-3f66-4c39-858d-93d3e468e87e admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:24.199276 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1166/2325] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:53:24 2022] PUT /identity/v3/projects/cf6fb66889b4489c9247928098504643/users/6fe52fbff8294575bed5a1dbb6cd78a9/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:53:24.209522 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aa5525-66d4-4359-97d6-3e07be65bbc6 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:24.209808 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aa5525-66d4-4359-97d6-3e07be65bbc6 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:24.210082 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aa5525-66d4-4359-97d6-3e07be65bbc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:24.241102 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-a0aa5525-66d4-4359-97d6-3e07be65bbc6 None None] MFA Rules not processed for user `6fe52fbff8294575bed5a1dbb6cd78a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:24.242740 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a0aa5525-66d4-4359-97d6-3e07be65bbc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:24.248072 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1160/2326] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:53:23 2022] POST /identity/v3/users => generated 288 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:53:24.252046 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b251b4-6cd6-4565-bc33-ee9c7e16541a admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:24.264664 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b251b4-6cd6-4565-bc33-ee9c7e16541a admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:24.334504 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059e3632-8e9e-4c5a-a574-7e28dddd8fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:24.335269 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059e3632-8e9e-4c5a-a574-7e28dddd8fe7 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:24.335549 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059e3632-8e9e-4c5a-a574-7e28dddd8fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:24.335827 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059e3632-8e9e-4c5a-a574-7e28dddd8fe7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:24.336602 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059e3632-8e9e-4c5a-a574-7e28dddd8fe7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:24.338536 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059e3632-8e9e-4c5a-a574-7e28dddd8fe7 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:24.340650 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1167/2327] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:53:24 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:53:24.351869 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aa5525-66d4-4359-97d6-3e07be65bbc6 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:24.353513 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a0aa5525-66d4-4359-97d6-3e07be65bbc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:24.355128 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1161/2328] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:53:24 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:53:24.372033 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059e3632-8e9e-4c5a-a574-7e28dddd8fe7 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:24.382942 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059e3632-8e9e-4c5a-a574-7e28dddd8fe7 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:24.389230 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b1e4c-0934-4f49-bd39-694819788be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:24.389664 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b1e4c-0934-4f49-bd39-694819788be2 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:24.389949 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b1e4c-0934-4f49-bd39-694819788be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:24.390226 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b1e4c-0934-4f49-bd39-694819788be2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:24.390955 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79b1e4c-0934-4f49-bd39-694819788be2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:24.392602 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79b1e4c-0934-4f49-bd39-694819788be2 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:24.403311 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1162/2329] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:53:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:53:24.408820 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b1e4c-0934-4f49-bd39-694819788be2 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:24.420172 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b1e4c-0934-4f49-bd39-694819788be2 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:24.420676 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aa5525-66d4-4359-97d6-3e07be65bbc6 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:24.425355 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866cb565-7411-4a93-be01-53da49486a87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:24.425647 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fb1c40-b0a8-4b94-8093-3d1b4c8149c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:24.426006 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866cb565-7411-4a93-be01-53da49486a87 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:24.426291 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866cb565-7411-4a93-be01-53da49486a87 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:24.426548 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fb1c40-b0a8-4b94-8093-3d1b4c8149c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:24.426653 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866cb565-7411-4a93-be01-53da49486a87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:24.426871 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fb1c40-b0a8-4b94-8093-3d1b4c8149c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:24.427205 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fb1c40-b0a8-4b94-8093-3d1b4c8149c7 admin admin] PATH_INFO: `/v3/projects/94c25ccc09364e679cdd6965f6e02e85/users/67e829b6db874ae8a0a7477ed23eec96/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:24.428441 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-866cb565-7411-4a93-be01-53da49486a87 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:24.445240 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fb1c40-b0a8-4b94-8093-3d1b4c8149c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94c25ccc09364e679cdd6965f6e02e85, user_id=67e829b6db874ae8a0a7477ed23eec96, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:24.448758 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fb1c40-b0a8-4b94-8093-3d1b4c8149c7 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:24.486284 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:24.493059 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866cb565-7411-4a93-be01-53da49486a87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:24.495037 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866cb565-7411-4a93-be01-53da49486a87 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:24.504569 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1168/2330] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:53:24 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:53:24.552130 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1163/2331] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:53:24 2022] PUT /identity/v3/projects/94c25ccc09364e679cdd6965f6e02e85/users/67e829b6db874ae8a0a7477ed23eec96/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:53:24.563217 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955f706b-cb83-4f9b-b0f9-cfd841ecb59d None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:24.563595 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955f706b-cb83-4f9b-b0f9-cfd841ecb59d None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:24.563877 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955f706b-cb83-4f9b-b0f9-cfd841ecb59d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:24.609179 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-955f706b-cb83-4f9b-b0f9-cfd841ecb59d None None] MFA Rules not processed for user `67e829b6db874ae8a0a7477ed23eec96`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:24.610694 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-955f706b-cb83-4f9b-b0f9-cfd841ecb59d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:24.685850 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1169/2332] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:53:24 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:53:24.704586 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fb1c40-b0a8-4b94-8093-3d1b4c8149c7 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:24.710562 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-59fb1c40-b0a8-4b94-8093-3d1b4c8149c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:24.772470 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fb1c40-b0a8-4b94-8093-3d1b4c8149c7 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:24.774371 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b12742-0d86-4874-80b0-cde7428a3da9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:24.775106 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b12742-0d86-4874-80b0-cde7428a3da9 service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:24.775387 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b12742-0d86-4874-80b0-cde7428a3da9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:24.775663 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b12742-0d86-4874-80b0-cde7428a3da9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:24.786601 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b12742-0d86-4874-80b0-cde7428a3da9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:24.788178 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b12742-0d86-4874-80b0-cde7428a3da9 service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:24.798988 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1164/2333] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:53:24 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:53:25.059577 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:26.264664 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955f706b-cb83-4f9b-b0f9-cfd841ecb59d None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:26.266937 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-955f706b-cb83-4f9b-b0f9-cfd841ecb59d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:26.354384 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955f706b-cb83-4f9b-b0f9-cfd841ecb59d None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:26.361457 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f2dc26-c61c-44ce-826c-4f791e38fbd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:26.365669 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f2dc26-c61c-44ce-826c-4f791e38fbd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:26.365669 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f2dc26-c61c-44ce-826c-4f791e38fbd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:26.365669 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f2dc26-c61c-44ce-826c-4f791e38fbd0 admin admin] PATH_INFO: `/v3/users/a2c9ea6873934c87bcdd07d5a6405e70` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:26.366497 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f2dc26-c61c-44ce-826c-4f791e38fbd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2c9ea6873934c87bcdd07d5a6405e70)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:26.370295 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f2dc26-c61c-44ce-826c-4f791e38fbd0 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:26.417467 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:26.614718 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-d5f2dc26-c61c-44ce-826c-4f791e38fbd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2c9ea6873934c87bcdd07d5a6405e70'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:26.615317 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-d5f2dc26-c61c-44ce-826c-4f791e38fbd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2c9ea6873934c87bcdd07d5a6405e70'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:26.615936 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-d5f2dc26-c61c-44ce-826c-4f791e38fbd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2c9ea6873934c87bcdd07d5a6405e70'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:26.687905 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-d5f2dc26-c61c-44ce-826c-4f791e38fbd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2c9ea6873934c87bcdd07d5a6405e70'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:26.702417 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-d5f2dc26-c61c-44ce-826c-4f791e38fbd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2c9ea6873934c87bcdd07d5a6405e70'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:26.715955 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1170/2334] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:53:26 2022] DELETE /identity/v3/users/a2c9ea6873934c87bcdd07d5a6405e70 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:53:26.750949 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:27.019943 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b12742-0d86-4874-80b0-cde7428a3da9 service cinder] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:27.031914 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b12742-0d86-4874-80b0-cde7428a3da9 service cinder] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:27.097207 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66ec590-2295-4a73-a781-2b23436bbc08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:27.098666 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ec590-2295-4a73-a781-2b23436bbc08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:27.099161 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ec590-2295-4a73-a781-2b23436bbc08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:27.099645 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ec590-2295-4a73-a781-2b23436bbc08 admin admin] PATH_INFO: `/v3/projects/f83130585c654ba6941c265f16e5eae9` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:27.101379 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66ec590-2295-4a73-a781-2b23436bbc08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f83130585c654ba6941c265f16e5eae9)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:27.104238 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66ec590-2295-4a73-a781-2b23436bbc08 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:27.216660 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-d66ec590-2295-4a73-a781-2b23436bbc08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f83130585c654ba6941c265f16e5eae9', 'request_id': 'req-d66ec590-2295-4a73-a781-2b23436bbc08'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:27.327487 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-d66ec590-2295-4a73-a781-2b23436bbc08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f83130585c654ba6941c265f16e5eae9', 'request_id': 'req-d66ec590-2295-4a73-a781-2b23436bbc08'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:27.334137 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-d66ec590-2295-4a73-a781-2b23436bbc08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f83130585c654ba6941c265f16e5eae9', 'request_id': 'req-d66ec590-2295-4a73-a781-2b23436bbc08'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:27.339630 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:27.346601 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-d66ec590-2295-4a73-a781-2b23436bbc08 admin admin] The token cache is being invalidate because project f83130585c654ba6941c265f16e5eae9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:53:27.349377 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-d66ec590-2295-4a73-a781-2b23436bbc08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:27.349377 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-d66ec590-2295-4a73-a781-2b23436bbc08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:27.349657 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1165/2335] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:53:27 2022] DELETE /identity/v3/projects/f83130585c654ba6941c265f16e5eae9 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:53:27.354357 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f2dc26-c61c-44ce-826c-4f791e38fbd0 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:27.355855 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-d5f2dc26-c61c-44ce-826c-4f791e38fbd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:27.370689 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f2dc26-c61c-44ce-826c-4f791e38fbd0 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:27.435996 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb063406-76bf-4cf8-a065-bd9a80ae58fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:27.437129 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb063406-76bf-4cf8-a065-bd9a80ae58fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:27.437482 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb063406-76bf-4cf8-a065-bd9a80ae58fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:27.437830 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb063406-76bf-4cf8-a065-bd9a80ae58fa admin admin] PATH_INFO: `/v3/users/25e3f6210dca4a3f967312eb7cdc8923` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:27.440126 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb063406-76bf-4cf8-a065-bd9a80ae58fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=25e3f6210dca4a3f967312eb7cdc8923)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:27.447344 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb063406-76bf-4cf8-a065-bd9a80ae58fa admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:27.710732 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:27.733804 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66ec590-2295-4a73-a781-2b23436bbc08 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:27.735273 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-d66ec590-2295-4a73-a781-2b23436bbc08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:27.760279 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-cb063406-76bf-4cf8-a065-bd9a80ae58fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25e3f6210dca4a3f967312eb7cdc8923'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:27.761086 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-cb063406-76bf-4cf8-a065-bd9a80ae58fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25e3f6210dca4a3f967312eb7cdc8923'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:27.761728 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-cb063406-76bf-4cf8-a065-bd9a80ae58fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25e3f6210dca4a3f967312eb7cdc8923'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:27.796496 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66ec590-2295-4a73-a781-2b23436bbc08 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:27.800195 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf59d4a-d123-4293-a43e-6b9067f04c84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:27.800856 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf59d4a-d123-4293-a43e-6b9067f04c84 service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:27.801143 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf59d4a-d123-4293-a43e-6b9067f04c84 service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:27.801422 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf59d4a-d123-4293-a43e-6b9067f04c84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:27.803275 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-1cf59d4a-d123-4293-a43e-6b9067f04c84 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:27.872854 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf59d4a-d123-4293-a43e-6b9067f04c84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:27.874322 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf59d4a-d123-4293-a43e-6b9067f04c84 service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:27.886729 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1166/2336] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:53:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:53:27.886989 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-cb063406-76bf-4cf8-a065-bd9a80ae58fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25e3f6210dca4a3f967312eb7cdc8923'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:27.902745 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-cb063406-76bf-4cf8-a065-bd9a80ae58fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25e3f6210dca4a3f967312eb7cdc8923'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:27.917025 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1171/2337] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:53:27 2022] DELETE /identity/v3/users/25e3f6210dca4a3f967312eb7cdc8923 => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:53:28.187807 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf59d4a-d123-4293-a43e-6b9067f04c84 service cinder] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:28.189411 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-1cf59d4a-d123-4293-a43e-6b9067f04c84 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:28.204915 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf59d4a-d123-4293-a43e-6b9067f04c84 service cinder] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:28.273775 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d05a15-e45d-4e5c-82d6-a378fd178f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:28.274613 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d05a15-e45d-4e5c-82d6-a378fd178f82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:28.279381 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d05a15-e45d-4e5c-82d6-a378fd178f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:28.279381 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d05a15-e45d-4e5c-82d6-a378fd178f82 admin admin] PATH_INFO: `/v3/projects/10f6f219518446f780c46b1fe41b062c` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:28.279381 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d05a15-e45d-4e5c-82d6-a378fd178f82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10f6f219518446f780c46b1fe41b062c)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:28.279381 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d05a15-e45d-4e5c-82d6-a378fd178f82 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:28.495768 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-53d05a15-e45d-4e5c-82d6-a378fd178f82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10f6f219518446f780c46b1fe41b062c', 'request_id': 'req-53d05a15-e45d-4e5c-82d6-a378fd178f82'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:28.557374 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-53d05a15-e45d-4e5c-82d6-a378fd178f82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10f6f219518446f780c46b1fe41b062c', 'request_id': 'req-53d05a15-e45d-4e5c-82d6-a378fd178f82'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:28.564909 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-53d05a15-e45d-4e5c-82d6-a378fd178f82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10f6f219518446f780c46b1fe41b062c', 'request_id': 'req-53d05a15-e45d-4e5c-82d6-a378fd178f82'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:28.581615 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-53d05a15-e45d-4e5c-82d6-a378fd178f82 admin admin] The token cache is being invalidate because project 10f6f219518446f780c46b1fe41b062c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:53:28.581915 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-53d05a15-e45d-4e5c-82d6-a378fd178f82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:28.582622 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-53d05a15-e45d-4e5c-82d6-a378fd178f82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:28.584472 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1167/2338] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:53:28 2022] DELETE /identity/v3/projects/10f6f219518446f780c46b1fe41b062c => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:53:28.704008 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:29.677607 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb063406-76bf-4cf8-a065-bd9a80ae58fa admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:29.679102 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-cb063406-76bf-4cf8-a065-bd9a80ae58fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:29.755996 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb063406-76bf-4cf8-a065-bd9a80ae58fa admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:29.803882 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:29.829039 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57acdf5d-731f-4d20-afef-e58721fcd671 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:29.829955 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57acdf5d-731f-4d20-afef-e58721fcd671 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:29.830374 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57acdf5d-731f-4d20-afef-e58721fcd671 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:29.830788 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57acdf5d-731f-4d20-afef-e58721fcd671 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:29.834198 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-57acdf5d-731f-4d20-afef-e58721fcd671 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:29.905021 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57acdf5d-731f-4d20-afef-e58721fcd671 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:29.906588 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57acdf5d-731f-4d20-afef-e58721fcd671 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:29.918098 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1172/2339] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:53:29 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:53:35.969461 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d05a15-e45d-4e5c-82d6-a378fd178f82 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:35.971732 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-53d05a15-e45d-4e5c-82d6-a378fd178f82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:36.047439 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d05a15-e45d-4e5c-82d6-a378fd178f82 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:36.047439 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e04e6e-cee2-410d-9536-1d82bf7458a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:36.047439 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e04e6e-cee2-410d-9536-1d82bf7458a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:36.047439 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e04e6e-cee2-410d-9536-1d82bf7458a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:36.047739 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e04e6e-cee2-410d-9536-1d82bf7458a5 admin admin] PATH_INFO: `/v3/users/66cc9f37bc2d4634875587ed1b2bf774` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:36.048520 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e04e6e-cee2-410d-9536-1d82bf7458a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66cc9f37bc2d4634875587ed1b2bf774)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:36.050438 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e04e6e-cee2-410d-9536-1d82bf7458a5 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:36.254034 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-46e04e6e-cee2-410d-9536-1d82bf7458a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66cc9f37bc2d4634875587ed1b2bf774'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:36.254723 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-46e04e6e-cee2-410d-9536-1d82bf7458a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66cc9f37bc2d4634875587ed1b2bf774'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:36.263302 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-46e04e6e-cee2-410d-9536-1d82bf7458a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66cc9f37bc2d4634875587ed1b2bf774'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:36.264033 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-46e04e6e-cee2-410d-9536-1d82bf7458a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66cc9f37bc2d4634875587ed1b2bf774'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:36.325309 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-46e04e6e-cee2-410d-9536-1d82bf7458a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66cc9f37bc2d4634875587ed1b2bf774'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:36.341184 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:36.346687 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1168/2340] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:53:35 2022] DELETE /identity/v3/users/66cc9f37bc2d4634875587ed1b2bf774 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:53:36.351390 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5145e-1bc0-4a49-81df-71e72857c923 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:36.351946 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5145e-1bc0-4a49-81df-71e72857c923 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:36.352041 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5145e-1bc0-4a49-81df-71e72857c923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:36.376847 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-46b5145e-1bc0-4a49-81df-71e72857c923 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:36.378146 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-46b5145e-1bc0-4a49-81df-71e72857c923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:36.441890 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1173/2341] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:53:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:53:36.451739 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e04e6e-cee2-410d-9536-1d82bf7458a5 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:36.453238 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-46e04e6e-cee2-410d-9536-1d82bf7458a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:36.514589 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e04e6e-cee2-410d-9536-1d82bf7458a5 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:36.518251 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccc9f24-5651-46ab-9365-229eff4fb51f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:36.518910 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccc9f24-5651-46ab-9365-229eff4fb51f service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:36.519198 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccc9f24-5651-46ab-9365-229eff4fb51f service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:36.519480 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccc9f24-5651-46ab-9365-229eff4fb51f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:36.530027 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccc9f24-5651-46ab-9365-229eff4fb51f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:36.533953 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccc9f24-5651-46ab-9365-229eff4fb51f service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:36.541654 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1169/2342] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:53:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:53:37.023689 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b5145e-1bc0-4a49-81df-71e72857c923 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:37.025300 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-46b5145e-1bc0-4a49-81df-71e72857c923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:37.043165 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b5145e-1bc0-4a49-81df-71e72857c923 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:37.048403 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f5f2aa-04e8-45cf-bade-7d9778cb6d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:37.049306 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f5f2aa-04e8-45cf-bade-7d9778cb6d59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:37.049592 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f5f2aa-04e8-45cf-bade-7d9778cb6d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:37.049874 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f5f2aa-04e8-45cf-bade-7d9778cb6d59 admin admin] PATH_INFO: `/v3/projects/b2cdfc40cdd4460f8d6bae576e144ec2` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:37.051135 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f5f2aa-04e8-45cf-bade-7d9778cb6d59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2cdfc40cdd4460f8d6bae576e144ec2)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:37.053112 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f5f2aa-04e8-45cf-bade-7d9778cb6d59 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:37.161657 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-46f5f2aa-04e8-45cf-bade-7d9778cb6d59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2cdfc40cdd4460f8d6bae576e144ec2', 'request_id': 'req-46f5f2aa-04e8-45cf-bade-7d9778cb6d59'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:37.168188 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-46f5f2aa-04e8-45cf-bade-7d9778cb6d59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2cdfc40cdd4460f8d6bae576e144ec2', 'request_id': 'req-46f5f2aa-04e8-45cf-bade-7d9778cb6d59'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:37.245329 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-46f5f2aa-04e8-45cf-bade-7d9778cb6d59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2cdfc40cdd4460f8d6bae576e144ec2', 'request_id': 'req-46f5f2aa-04e8-45cf-bade-7d9778cb6d59'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:37.256460 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-46f5f2aa-04e8-45cf-bade-7d9778cb6d59 admin admin] The token cache is being invalidate because project b2cdfc40cdd4460f8d6bae576e144ec2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:53:37.256749 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-46f5f2aa-04e8-45cf-bade-7d9778cb6d59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:37.257403 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-46f5f2aa-04e8-45cf-bade-7d9778cb6d59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:37.259028 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1174/2343] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:53:37 2022] DELETE /identity/v3/projects/b2cdfc40cdd4460f8d6bae576e144ec2 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:53:37.263309 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccc9f24-5651-46ab-9365-229eff4fb51f service neutron] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:37.275325 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccc9f24-5651-46ab-9365-229eff4fb51f service neutron] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:37.336233 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af66251f-6c8d-42dd-a568-cf38d14f863c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:37.337802 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af66251f-6c8d-42dd-a568-cf38d14f863c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:37.338274 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af66251f-6c8d-42dd-a568-cf38d14f863c admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:37.338754 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af66251f-6c8d-42dd-a568-cf38d14f863c admin admin] PATH_INFO: `/v3/users/1849b699f701423b91e98f3dad25ca96` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:37.340981 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af66251f-6c8d-42dd-a568-cf38d14f863c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1849b699f701423b91e98f3dad25ca96)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:37.344697 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af66251f-6c8d-42dd-a568-cf38d14f863c admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:37.527905 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-af66251f-6c8d-42dd-a568-cf38d14f863c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1849b699f701423b91e98f3dad25ca96'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:37.528487 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-af66251f-6c8d-42dd-a568-cf38d14f863c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1849b699f701423b91e98f3dad25ca96'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:37.535320 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-af66251f-6c8d-42dd-a568-cf38d14f863c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1849b699f701423b91e98f3dad25ca96'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:37.535955 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-af66251f-6c8d-42dd-a568-cf38d14f863c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1849b699f701423b91e98f3dad25ca96'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:37.601590 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-af66251f-6c8d-42dd-a568-cf38d14f863c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1849b699f701423b91e98f3dad25ca96'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:37.620664 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1170/2344] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:53:37 2022] DELETE /identity/v3/users/1849b699f701423b91e98f3dad25ca96 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:53:38.128796 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f5f2aa-04e8-45cf-bade-7d9778cb6d59 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:38.130267 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-46f5f2aa-04e8-45cf-bade-7d9778cb6d59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:38.140169 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:38.144296 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f5f2aa-04e8-45cf-bade-7d9778cb6d59 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:38.210399 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8881cf75-0664-473e-a36b-7c26762c79b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:38.211275 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881cf75-0664-473e-a36b-7c26762c79b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:38.211612 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881cf75-0664-473e-a36b-7c26762c79b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:38.211895 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881cf75-0664-473e-a36b-7c26762c79b4 admin admin] PATH_INFO: `/v3/projects/c50914d0345048fab34fd0ccfb91e190` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:38.213202 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8881cf75-0664-473e-a36b-7c26762c79b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c50914d0345048fab34fd0ccfb91e190)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:38.215152 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8881cf75-0664-473e-a36b-7c26762c79b4 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:38.346167 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8881cf75-0664-473e-a36b-7c26762c79b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c50914d0345048fab34fd0ccfb91e190', 'request_id': 'req-8881cf75-0664-473e-a36b-7c26762c79b4'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:38.352388 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8881cf75-0664-473e-a36b-7c26762c79b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c50914d0345048fab34fd0ccfb91e190', 'request_id': 'req-8881cf75-0664-473e-a36b-7c26762c79b4'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:38.453062 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8881cf75-0664-473e-a36b-7c26762c79b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c50914d0345048fab34fd0ccfb91e190', 'request_id': 'req-8881cf75-0664-473e-a36b-7c26762c79b4'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:38.466191 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8881cf75-0664-473e-a36b-7c26762c79b4 admin admin] The token cache is being invalidate because project c50914d0345048fab34fd0ccfb91e190 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:53:38.466632 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8881cf75-0664-473e-a36b-7c26762c79b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:38.467381 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8881cf75-0664-473e-a36b-7c26762c79b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:38.469166 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1175/2345] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:53:38 2022] DELETE /identity/v3/projects/c50914d0345048fab34fd0ccfb91e190 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:53:38.473616 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af66251f-6c8d-42dd-a568-cf38d14f863c admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:38.475292 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-af66251f-6c8d-42dd-a568-cf38d14f863c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:38.489071 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af66251f-6c8d-42dd-a568-cf38d14f863c admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:38.506554 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:38.547981 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe01450-11e0-429d-85f3-f8a3e2d5ee1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:38.548897 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe01450-11e0-429d-85f3-f8a3e2d5ee1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:38.549181 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe01450-11e0-429d-85f3-f8a3e2d5ee1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:38.549458 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe01450-11e0-429d-85f3-f8a3e2d5ee1b admin admin] PATH_INFO: `/v3/users/6fe52fbff8294575bed5a1dbb6cd78a9` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:38.550762 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe01450-11e0-429d-85f3-f8a3e2d5ee1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fe52fbff8294575bed5a1dbb6cd78a9)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:38.552750 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe01450-11e0-429d-85f3-f8a3e2d5ee1b admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:38.825753 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-6fe01450-11e0-429d-85f3-f8a3e2d5ee1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fe52fbff8294575bed5a1dbb6cd78a9'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:38.826420 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-6fe01450-11e0-429d-85f3-f8a3e2d5ee1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fe52fbff8294575bed5a1dbb6cd78a9'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:38.834568 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-6fe01450-11e0-429d-85f3-f8a3e2d5ee1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fe52fbff8294575bed5a1dbb6cd78a9'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:38.835274 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-6fe01450-11e0-429d-85f3-f8a3e2d5ee1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fe52fbff8294575bed5a1dbb6cd78a9'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:38.927391 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-6fe01450-11e0-429d-85f3-f8a3e2d5ee1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fe52fbff8294575bed5a1dbb6cd78a9'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:38.946909 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1171/2346] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:53:38 2022] DELETE /identity/v3/users/6fe52fbff8294575bed5a1dbb6cd78a9 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:53:39.380143 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8881cf75-0664-473e-a36b-7c26762c79b4 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:39.381683 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-8881cf75-0664-473e-a36b-7c26762c79b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:39.395539 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8881cf75-0664-473e-a36b-7c26762c79b4 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:39.454352 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36d132e-5810-4943-8515-eae4c808592d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:39.455204 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36d132e-5810-4943-8515-eae4c808592d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:39.457617 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36d132e-5810-4943-8515-eae4c808592d admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:39.457617 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36d132e-5810-4943-8515-eae4c808592d admin admin] PATH_INFO: `/v3/projects/cf6fb66889b4489c9247928098504643` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:39.457617 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36d132e-5810-4943-8515-eae4c808592d admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf6fb66889b4489c9247928098504643)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:39.458907 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36d132e-5810-4943-8515-eae4c808592d admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:39.542959 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:39.551783 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-a36d132e-5810-4943-8515-eae4c808592d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf6fb66889b4489c9247928098504643', 'request_id': 'req-a36d132e-5810-4943-8515-eae4c808592d'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:39.558113 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-a36d132e-5810-4943-8515-eae4c808592d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf6fb66889b4489c9247928098504643', 'request_id': 'req-a36d132e-5810-4943-8515-eae4c808592d'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:39.653936 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-a36d132e-5810-4943-8515-eae4c808592d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf6fb66889b4489c9247928098504643', 'request_id': 'req-a36d132e-5810-4943-8515-eae4c808592d'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:39.665166 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-a36d132e-5810-4943-8515-eae4c808592d admin admin] The token cache is being invalidate because project cf6fb66889b4489c9247928098504643 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:53:39.665454 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-a36d132e-5810-4943-8515-eae4c808592d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:39.666084 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-a36d132e-5810-4943-8515-eae4c808592d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:39.667869 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1176/2347] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:53:39 2022] DELETE /identity/v3/projects/cf6fb66889b4489c9247928098504643 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:53:39.673148 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a3039-b8bd-4787-ac45-ef9e4112b704 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:39.673603 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a3039-b8bd-4787-ac45-ef9e4112b704 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:39.673778 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a3039-b8bd-4787-ac45-ef9e4112b704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:39.692441 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-e70a3039-b8bd-4787-ac45-ef9e4112b704 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:39.693680 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-e70a3039-b8bd-4787-ac45-ef9e4112b704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:39.753520 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1172/2348] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:53:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:53:39.763249 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0457a0f9-906a-4eaa-9bd2-2935c16075f6 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:39.763535 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0457a0f9-906a-4eaa-9bd2-2935c16075f6 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:39.763805 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0457a0f9-906a-4eaa-9bd2-2935c16075f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:39.783011 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-0457a0f9-906a-4eaa-9bd2-2935c16075f6 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:39.784254 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-0457a0f9-906a-4eaa-9bd2-2935c16075f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:39.792410 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1177/2349] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:53:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:53:39.797275 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70a3039-b8bd-4787-ac45-ef9e4112b704 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:39.808823 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70a3039-b8bd-4787-ac45-ef9e4112b704 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:39.813516 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9272dd-291d-4897-9a4f-4942a73a6789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:39.814166 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9272dd-291d-4897-9a4f-4942a73a6789 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:39.814445 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9272dd-291d-4897-9a4f-4942a73a6789 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:39.814724 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9272dd-291d-4897-9a4f-4942a73a6789 admin admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:39.815526 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9272dd-291d-4897-9a4f-4942a73a6789 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:53:39.815828 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9272dd-291d-4897-9a4f-4942a73a6789 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:39.817590 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9272dd-291d-4897-9a4f-4942a73a6789 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:39.831164 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1173/2350] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jun 15 14:53:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:53:39.835570 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0457a0f9-906a-4eaa-9bd2-2935c16075f6 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:39.845489 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0457a0f9-906a-4eaa-9bd2-2935c16075f6 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:39.850299 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5956d27f-1ecc-45bb-b343-300ae010377b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:39.850958 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5956d27f-1ecc-45bb-b343-300ae010377b admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:39.851243 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5956d27f-1ecc-45bb-b343-300ae010377b admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:39.851522 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5956d27f-1ecc-45bb-b343-300ae010377b admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:39.854388 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5956d27f-1ecc-45bb-b343-300ae010377b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:39.854388 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5956d27f-1ecc-45bb-b343-300ae010377b admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:39.942159 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1178/2351] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:53:39 2022] POST /identity/v3/projects => generated 371 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:53:39.943456 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9272dd-291d-4897-9a4f-4942a73a6789 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:39.954328 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9272dd-291d-4897-9a4f-4942a73a6789 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:39.979423 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:40.012661 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b971e7-bdf2-4602-88fa-53ca8a36b458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:40.013344 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b971e7-bdf2-4602-88fa-53ca8a36b458 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:40.013628 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b971e7-bdf2-4602-88fa-53ca8a36b458 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:40.013907 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b971e7-bdf2-4602-88fa-53ca8a36b458 admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:40.014612 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b971e7-bdf2-4602-88fa-53ca8a36b458 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:40.016644 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b971e7-bdf2-4602-88fa-53ca8a36b458 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:40.108493 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1174/2352] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:53:39 2022] POST /identity/v3/users => generated 286 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:53:40.112466 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5956d27f-1ecc-45bb-b343-300ae010377b admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:40.124299 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5956d27f-1ecc-45bb-b343-300ae010377b admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:40.129038 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c957cd03-c3e6-4c1d-980d-e8c2eb5c636c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:40.129760 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c957cd03-c3e6-4c1d-980d-e8c2eb5c636c admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:40.130091 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c957cd03-c3e6-4c1d-980d-e8c2eb5c636c admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:40.130365 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c957cd03-c3e6-4c1d-980d-e8c2eb5c636c admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:40.131098 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c957cd03-c3e6-4c1d-980d-e8c2eb5c636c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:40.132728 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c957cd03-c3e6-4c1d-980d-e8c2eb5c636c admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:40.144936 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1179/2353] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:53:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:53:40.149087 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b971e7-bdf2-4602-88fa-53ca8a36b458 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:40.159449 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b971e7-bdf2-4602-88fa-53ca8a36b458 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:40.164626 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7b14cf-f4a4-41bb-abda-4a9b3d61f00e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:40.165302 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7b14cf-f4a4-41bb-abda-4a9b3d61f00e admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:40.165585 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7b14cf-f4a4-41bb-abda-4a9b3d61f00e admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:40.165862 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7b14cf-f4a4-41bb-abda-4a9b3d61f00e admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:40.166586 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7b14cf-f4a4-41bb-abda-4a9b3d61f00e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:40.169083 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7b14cf-f4a4-41bb-abda-4a9b3d61f00e admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:40.179747 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1175/2354] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:53:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:53:40.185012 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c957cd03-c3e6-4c1d-980d-e8c2eb5c636c admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:40.195363 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c957cd03-c3e6-4c1d-980d-e8c2eb5c636c admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:40.200555 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef877042-fadd-40d7-9cf6-897d0331c280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:40.201477 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef877042-fadd-40d7-9cf6-897d0331c280 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:40.201756 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef877042-fadd-40d7-9cf6-897d0331c280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:40.202031 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef877042-fadd-40d7-9cf6-897d0331c280 admin admin] PATH_INFO: `/v3/projects/f3340f5c4bac49cfaaaa0d80b1bfbb4a/users/a0e682af1a7f493c992509142b673932/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:40.217359 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef877042-fadd-40d7-9cf6-897d0331c280 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3340f5c4bac49cfaaaa0d80b1bfbb4a, user_id=a0e682af1a7f493c992509142b673932, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:40.220021 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef877042-fadd-40d7-9cf6-897d0331c280 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:40.288264 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7b14cf-f4a4-41bb-abda-4a9b3d61f00e admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:40.290025 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-da7b14cf-f4a4-41bb-abda-4a9b3d61f00e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:40.303620 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7b14cf-f4a4-41bb-abda-4a9b3d61f00e admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:40.309002 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e877b-4d9a-4013-acb8-5a310d00e263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:40.309885 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e877b-4d9a-4013-acb8-5a310d00e263 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:40.310171 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e877b-4d9a-4013-acb8-5a310d00e263 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:40.310452 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e877b-4d9a-4013-acb8-5a310d00e263 admin admin] PATH_INFO: `/v3/users/67e829b6db874ae8a0a7477ed23eec96` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:40.311665 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4e877b-4d9a-4013-acb8-5a310d00e263 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67e829b6db874ae8a0a7477ed23eec96)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:40.313619 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4e877b-4d9a-4013-acb8-5a310d00e263 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:40.322284 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1180/2355] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:53:40 2022] PUT /identity/v3/projects/f3340f5c4bac49cfaaaa0d80b1bfbb4a/users/a0e682af1a7f493c992509142b673932/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:53:40.327364 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa73773b-c0ae-4620-b9e9-3161a37eafa5 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:40.327654 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa73773b-c0ae-4620-b9e9-3161a37eafa5 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:40.327940 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa73773b-c0ae-4620-b9e9-3161a37eafa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:40.345767 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-fa73773b-c0ae-4620-b9e9-3161a37eafa5 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:40.346997 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-fa73773b-c0ae-4620-b9e9-3161a37eafa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:40.414120 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1181/2356] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:53:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:53:40.424225 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa73773b-c0ae-4620-b9e9-3161a37eafa5 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:40.425663 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-fa73773b-c0ae-4620-b9e9-3161a37eafa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:40.489114 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa73773b-c0ae-4620-b9e9-3161a37eafa5 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:40.492974 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcf1f7e-85d2-46b0-8367-539d41661136 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:40.493628 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcf1f7e-85d2-46b0-8367-539d41661136 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:40.493909 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcf1f7e-85d2-46b0-8367-539d41661136 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:40.494185 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcf1f7e-85d2-46b0-8367-539d41661136 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:40.504765 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcf1f7e-85d2-46b0-8367-539d41661136 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:40.506223 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcf1f7e-85d2-46b0-8367-539d41661136 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:40.513890 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-6e4e877b-4d9a-4013-acb8-5a310d00e263 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67e829b6db874ae8a0a7477ed23eec96'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:40.514644 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-6e4e877b-4d9a-4013-acb8-5a310d00e263 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67e829b6db874ae8a0a7477ed23eec96'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:40.518886 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1182/2357] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:53:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:53:40.525655 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-6e4e877b-4d9a-4013-acb8-5a310d00e263 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67e829b6db874ae8a0a7477ed23eec96'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:40.527023 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-6e4e877b-4d9a-4013-acb8-5a310d00e263 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67e829b6db874ae8a0a7477ed23eec96'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:40.596142 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-6e4e877b-4d9a-4013-acb8-5a310d00e263 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67e829b6db874ae8a0a7477ed23eec96'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:40.614441 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1176/2358] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:53:40 2022] DELETE /identity/v3/users/67e829b6db874ae8a0a7477ed23eec96 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:53:40.619073 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9348db-b5ad-41af-8809-23f2e4f473f8 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:40.619386 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9348db-b5ad-41af-8809-23f2e4f473f8 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:40.619642 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9348db-b5ad-41af-8809-23f2e4f473f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:40.627036 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:40.640204 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-3e9348db-b5ad-41af-8809-23f2e4f473f8 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:40.641462 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-3e9348db-b5ad-41af-8809-23f2e4f473f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:40.704864 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1183/2359] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:53:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:53:40.714745 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e877b-4d9a-4013-acb8-5a310d00e263 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:40.720038 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-6e4e877b-4d9a-4013-acb8-5a310d00e263 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:40.774373 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e877b-4d9a-4013-acb8-5a310d00e263 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:40.778214 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023297f3-35d8-400b-a3a5-4ce0842149f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:40.778859 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023297f3-35d8-400b-a3a5-4ce0842149f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:40.779142 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023297f3-35d8-400b-a3a5-4ce0842149f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:40.779418 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023297f3-35d8-400b-a3a5-4ce0842149f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:40.788541 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023297f3-35d8-400b-a3a5-4ce0842149f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:40.790010 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023297f3-35d8-400b-a3a5-4ce0842149f3 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:40.799677 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1177/2360] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:53:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:53:41.264786 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9348db-b5ad-41af-8809-23f2e4f473f8 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:41.266295 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-3e9348db-b5ad-41af-8809-23f2e4f473f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:41.280686 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9348db-b5ad-41af-8809-23f2e4f473f8 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:41.286053 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f418f90-c82c-4b30-bbcb-0735faf56c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:41.287070 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f418f90-c82c-4b30-bbcb-0735faf56c1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:41.287477 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f418f90-c82c-4b30-bbcb-0735faf56c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:41.287854 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f418f90-c82c-4b30-bbcb-0735faf56c1b admin admin] PATH_INFO: `/v3/projects/94c25ccc09364e679cdd6965f6e02e85` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:41.289298 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f418f90-c82c-4b30-bbcb-0735faf56c1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=94c25ccc09364e679cdd6965f6e02e85)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:41.291320 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f418f90-c82c-4b30-bbcb-0735faf56c1b admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:41.422266 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8f418f90-c82c-4b30-bbcb-0735faf56c1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94c25ccc09364e679cdd6965f6e02e85', 'request_id': 'req-8f418f90-c82c-4b30-bbcb-0735faf56c1b'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:41.430275 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8f418f90-c82c-4b30-bbcb-0735faf56c1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94c25ccc09364e679cdd6965f6e02e85', 'request_id': 'req-8f418f90-c82c-4b30-bbcb-0735faf56c1b'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:41.498892 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8f418f90-c82c-4b30-bbcb-0735faf56c1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94c25ccc09364e679cdd6965f6e02e85', 'request_id': 'req-8f418f90-c82c-4b30-bbcb-0735faf56c1b'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:41.511525 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8f418f90-c82c-4b30-bbcb-0735faf56c1b admin admin] The token cache is being invalidate because project 94c25ccc09364e679cdd6965f6e02e85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:53:41.512121 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8f418f90-c82c-4b30-bbcb-0735faf56c1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:41.512953 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8f418f90-c82c-4b30-bbcb-0735faf56c1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:41.514723 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1184/2361] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:53:41 2022] DELETE /identity/v3/projects/94c25ccc09364e679cdd6965f6e02e85 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:53:41.520674 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb586075-9cc6-4985-b921-a9e13f12c0cd None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:41.520955 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb586075-9cc6-4985-b921-a9e13f12c0cd None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:41.521233 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb586075-9cc6-4985-b921-a9e13f12c0cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:41.544780 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-bb586075-9cc6-4985-b921-a9e13f12c0cd None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:41.546445 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-bb586075-9cc6-4985-b921-a9e13f12c0cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:41.562134 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:41.615788 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1178/2362] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:53:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:53:41.624496 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c312d6-de58-4403-aeb2-d399063419f6 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:41.624781 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c312d6-de58-4403-aeb2-d399063419f6 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:41.625060 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c312d6-de58-4403-aeb2-d399063419f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:41.644390 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-34c312d6-de58-4403-aeb2-d399063419f6 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:41.645647 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-34c312d6-de58-4403-aeb2-d399063419f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:41.654714 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1185/2363] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:53:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:53:41.659355 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb586075-9cc6-4985-b921-a9e13f12c0cd None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:41.670121 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb586075-9cc6-4985-b921-a9e13f12c0cd None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:41.675289 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94239815-0149-4f92-bed9-d67cd4d78b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:41.675975 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94239815-0149-4f92-bed9-d67cd4d78b71 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:41.677074 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94239815-0149-4f92-bed9-d67cd4d78b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:41.677074 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94239815-0149-4f92-bed9-d67cd4d78b71 admin admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:41.677994 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94239815-0149-4f92-bed9-d67cd4d78b71 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:53:41.678414 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94239815-0149-4f92-bed9-d67cd4d78b71 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:41.680011 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94239815-0149-4f92-bed9-d67cd4d78b71 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:41.693262 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1179/2364] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jun 15 14:53:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:53:41.697434 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c312d6-de58-4403-aeb2-d399063419f6 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:41.707509 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c312d6-de58-4403-aeb2-d399063419f6 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:41.712650 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776ab82c-8531-420d-b080-7ccf92797ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:41.713312 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776ab82c-8531-420d-b080-7ccf92797ab2 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:41.718707 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776ab82c-8531-420d-b080-7ccf92797ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:41.718707 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776ab82c-8531-420d-b080-7ccf92797ab2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:41.718707 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776ab82c-8531-420d-b080-7ccf92797ab2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:41.718707 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776ab82c-8531-420d-b080-7ccf92797ab2 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:41.847757 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1186/2365] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:53:41 2022] POST /identity/v3/projects => generated 373 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:53:41.851850 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94239815-0149-4f92-bed9-d67cd4d78b71 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:41.862028 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94239815-0149-4f92-bed9-d67cd4d78b71 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:41.923757 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2c00d3-3971-4a7b-a487-a1f109bf17c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:41.924463 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c00d3-3971-4a7b-a487-a1f109bf17c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:41.924745 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c00d3-3971-4a7b-a487-a1f109bf17c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:41.925022 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c00d3-3971-4a7b-a487-a1f109bf17c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:41.925709 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2c00d3-3971-4a7b-a487-a1f109bf17c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:41.927624 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2c00d3-3971-4a7b-a487-a1f109bf17c0 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:42.023523 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1180/2366] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:53:41 2022] POST /identity/v3/users => generated 287 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:53:42.028265 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776ab82c-8531-420d-b080-7ccf92797ab2 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:42.039468 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776ab82c-8531-420d-b080-7ccf92797ab2 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:42.045387 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0a01fc-9437-427f-9419-35fabad62838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:42.046105 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0a01fc-9437-427f-9419-35fabad62838 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:42.046538 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0a01fc-9437-427f-9419-35fabad62838 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:42.046826 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0a01fc-9437-427f-9419-35fabad62838 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:42.047616 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0a01fc-9437-427f-9419-35fabad62838 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:42.049319 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0a01fc-9437-427f-9419-35fabad62838 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:42.062380 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1187/2367] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:53:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:53:42.066519 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2c00d3-3971-4a7b-a487-a1f109bf17c0 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:42.076839 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2c00d3-3971-4a7b-a487-a1f109bf17c0 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:42.081884 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d30a3a-13ff-40ac-9ec4-f2fc8cc9ccec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:42.082680 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d30a3a-13ff-40ac-9ec4-f2fc8cc9ccec admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:42.089122 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d30a3a-13ff-40ac-9ec4-f2fc8cc9ccec admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:42.089122 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d30a3a-13ff-40ac-9ec4-f2fc8cc9ccec admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:42.089122 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d30a3a-13ff-40ac-9ec4-f2fc8cc9ccec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:42.089122 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d30a3a-13ff-40ac-9ec4-f2fc8cc9ccec admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:42.099141 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1181/2368] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:53:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:53:42.103198 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0a01fc-9437-427f-9419-35fabad62838 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:42.117576 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0a01fc-9437-427f-9419-35fabad62838 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:42.128072 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbb4cee-b8ac-4174-989a-68fa2c814464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:42.129107 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbb4cee-b8ac-4174-989a-68fa2c814464 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:42.129512 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbb4cee-b8ac-4174-989a-68fa2c814464 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:42.129863 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbb4cee-b8ac-4174-989a-68fa2c814464 admin admin] PATH_INFO: `/v3/projects/2addbe652f344f1583f0198fbd62864d/users/be81f5426d684c95b21bf45ffbca87a2/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:42.148815 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbb4cee-b8ac-4174-989a-68fa2c814464 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2addbe652f344f1583f0198fbd62864d, user_id=be81f5426d684c95b21bf45ffbca87a2, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:42.152269 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbb4cee-b8ac-4174-989a-68fa2c814464 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:42.240851 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1188/2369] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:53:42 2022] PUT /identity/v3/projects/2addbe652f344f1583f0198fbd62864d/users/be81f5426d684c95b21bf45ffbca87a2/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:53:42.251721 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72735702-2072-4a5c-9d62-b0adc0523289 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:42.251998 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72735702-2072-4a5c-9d62-b0adc0523289 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:42.252310 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72735702-2072-4a5c-9d62-b0adc0523289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:42.284384 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-72735702-2072-4a5c-9d62-b0adc0523289 None None] MFA Rules not processed for user `be81f5426d684c95b21bf45ffbca87a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:42.285698 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-72735702-2072-4a5c-9d62-b0adc0523289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:42.359801 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1182/2370] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:53:42 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:53:42.372139 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbb4cee-b8ac-4174-989a-68fa2c814464 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:42.373832 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-5bbb4cee-b8ac-4174-989a-68fa2c814464 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:42.407792 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:42.441306 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbb4cee-b8ac-4174-989a-68fa2c814464 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:42.445259 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba085bf-ff0f-46fd-a7c8-e9ecb1b5da7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:42.445923 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba085bf-ff0f-46fd-a7c8-e9ecb1b5da7d service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:42.446211 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba085bf-ff0f-46fd-a7c8-e9ecb1b5da7d service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:42.446491 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba085bf-ff0f-46fd-a7c8-e9ecb1b5da7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:42.457057 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba085bf-ff0f-46fd-a7c8-e9ecb1b5da7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:42.458586 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba085bf-ff0f-46fd-a7c8-e9ecb1b5da7d service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:42.469448 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1189/2371] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:53:42 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:53:45.417629 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f146000-fd52-4572-b926-034524b3c672 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:45.418131 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f146000-fd52-4572-b926-034524b3c672 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:45.418515 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f146000-fd52-4572-b926-034524b3c672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:45.438830 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-7f146000-fd52-4572-b926-034524b3c672 None None] MFA Rules not processed for user `be81f5426d684c95b21bf45ffbca87a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:45.440334 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-7f146000-fd52-4572-b926-034524b3c672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:45.449676 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1183/2372] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:53:45 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:53:46.587414 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6800689e-60f5-451f-9c4d-bd3a59ddcc07 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:46.587957 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6800689e-60f5-451f-9c4d-bd3a59ddcc07 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:46.587957 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6800689e-60f5-451f-9c4d-bd3a59ddcc07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:46.623009 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-6800689e-60f5-451f-9c4d-bd3a59ddcc07 None None] MFA Rules not processed for user `a0e682af1a7f493c992509142b673932`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:46.624786 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-6800689e-60f5-451f-9c4d-bd3a59ddcc07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:46.713046 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1190/2373] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:53:46 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:53:46.718108 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f146000-fd52-4572-b926-034524b3c672 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:46.722633 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-7f146000-fd52-4572-b926-034524b3c672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:46.737686 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:46.812219 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f146000-fd52-4572-b926-034524b3c672 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:46.817217 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40db75b9-7d2c-4d36-8c52-275e7a1c89e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:46.817928 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40db75b9-7d2c-4d36-8c52-275e7a1c89e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:46.818219 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40db75b9-7d2c-4d36-8c52-275e7a1c89e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:46.818536 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40db75b9-7d2c-4d36-8c52-275e7a1c89e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:46.819518 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40db75b9-7d2c-4d36-8c52-275e7a1c89e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:46.821477 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40db75b9-7d2c-4d36-8c52-275e7a1c89e8 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:46.936330 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1184/2374] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:53:46 2022] POST /identity/v3/projects => generated 373 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:53:46.941125 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6800689e-60f5-451f-9c4d-bd3a59ddcc07 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:46.954527 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6800689e-60f5-451f-9c4d-bd3a59ddcc07 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:47.025209 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ca7e2a-ad8e-4645-9af6-a544e2ff0d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:47.026636 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ca7e2a-ad8e-4645-9af6-a544e2ff0d15 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:47.027010 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ca7e2a-ad8e-4645-9af6-a544e2ff0d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:47.027289 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ca7e2a-ad8e-4645-9af6-a544e2ff0d15 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:47.028000 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ca7e2a-ad8e-4645-9af6-a544e2ff0d15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:47.031853 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ca7e2a-ad8e-4645-9af6-a544e2ff0d15 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:47.157564 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1191/2375] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:53:46 2022] POST /identity/v3/users => generated 293 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:53:47.161718 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40db75b9-7d2c-4d36-8c52-275e7a1c89e8 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:47.173762 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40db75b9-7d2c-4d36-8c52-275e7a1c89e8 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:47.178717 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a9583b-89c9-420e-859b-b8b0ab8ece14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:47.179411 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a9583b-89c9-420e-859b-b8b0ab8ece14 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:47.179703 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a9583b-89c9-420e-859b-b8b0ab8ece14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:47.179981 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a9583b-89c9-420e-859b-b8b0ab8ece14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:47.180752 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a9583b-89c9-420e-859b-b8b0ab8ece14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:47.183227 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a9583b-89c9-420e-859b-b8b0ab8ece14 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:47.195886 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1185/2376] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:53:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:53:47.201034 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ca7e2a-ad8e-4645-9af6-a544e2ff0d15 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:47.214608 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ca7e2a-ad8e-4645-9af6-a544e2ff0d15 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:47.220022 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e4ac24-7280-4a35-b98d-c1eb923255e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:47.220963 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e4ac24-7280-4a35-b98d-c1eb923255e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:47.221990 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e4ac24-7280-4a35-b98d-c1eb923255e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:47.222648 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e4ac24-7280-4a35-b98d-c1eb923255e5 admin admin] PATH_INFO: `/v3/projects/646ee05411a1467988f0ad556741c05e/users/9a3d6ad33aeb440b821f6efc69a39671/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:47.242149 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e4ac24-7280-4a35-b98d-c1eb923255e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=646ee05411a1467988f0ad556741c05e, user_id=9a3d6ad33aeb440b821f6efc69a39671, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:47.246016 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e4ac24-7280-4a35-b98d-c1eb923255e5 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:47.340480 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1192/2377] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:53:47 2022] PUT /identity/v3/projects/646ee05411a1467988f0ad556741c05e/users/9a3d6ad33aeb440b821f6efc69a39671/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:53:47.344457 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a9583b-89c9-420e-859b-b8b0ab8ece14 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:47.356569 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a9583b-89c9-420e-859b-b8b0ab8ece14 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:47.423889 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2156d3ed-1d0d-42da-a58e-157a9e9861a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:47.424682 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156d3ed-1d0d-42da-a58e-157a9e9861a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:47.425037 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156d3ed-1d0d-42da-a58e-157a9e9861a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:47.425386 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156d3ed-1d0d-42da-a58e-157a9e9861a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:47.426228 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2156d3ed-1d0d-42da-a58e-157a9e9861a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:47.427950 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2156d3ed-1d0d-42da-a58e-157a9e9861a5 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:47.439589 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1186/2378] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:53:47 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:53:47.443900 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e4ac24-7280-4a35-b98d-c1eb923255e5 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:47.456718 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e4ac24-7280-4a35-b98d-c1eb923255e5 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:47.461582 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b88ce5-e652-4223-b4a9-6ba663e6be88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:47.462496 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b88ce5-e652-4223-b4a9-6ba663e6be88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:47.462785 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b88ce5-e652-4223-b4a9-6ba663e6be88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:47.463065 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b88ce5-e652-4223-b4a9-6ba663e6be88 admin admin] PATH_INFO: `/v3/projects/646ee05411a1467988f0ad556741c05e/users/9a3d6ad33aeb440b821f6efc69a39671/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:47.465832 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b88ce5-e652-4223-b4a9-6ba663e6be88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=646ee05411a1467988f0ad556741c05e, user_id=9a3d6ad33aeb440b821f6efc69a39671, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:47.468720 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b88ce5-e652-4223-b4a9-6ba663e6be88 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:47.481055 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1193/2379] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:53:47 2022] PUT /identity/v3/projects/646ee05411a1467988f0ad556741c05e/users/9a3d6ad33aeb440b821f6efc69a39671/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:53:54.628803 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e20b89a-7534-4459-93d8-56139ad378d2 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:54.629958 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e20b89a-7534-4459-93d8-56139ad378d2 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:54.630325 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e20b89a-7534-4459-93d8-56139ad378d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:54.669787 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-0e20b89a-7534-4459-93d8-56139ad378d2 None None] MFA Rules not processed for user `9a3d6ad33aeb440b821f6efc69a39671`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:54.672008 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-0e20b89a-7534-4459-93d8-56139ad378d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:54.786823 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1187/2380] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:53:54 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:53:54.802600 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b88ce5-e652-4223-b4a9-6ba663e6be88 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:54.804731 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-64b88ce5-e652-4223-b4a9-6ba663e6be88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:54.820285 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:54.914338 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b88ce5-e652-4223-b4a9-6ba663e6be88 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:54.923977 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce57d44-a77f-42cc-8176-03bb1596469c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0927c05c340c48e48b18754888aa9c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:54.925148 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce57d44-a77f-42cc-8176-03bb1596469c service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:54.925642 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce57d44-a77f-42cc-8176-03bb1596469c service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:54.926146 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce57d44-a77f-42cc-8176-03bb1596469c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:55.013348 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce57d44-a77f-42cc-8176-03bb1596469c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:55.015066 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce57d44-a77f-42cc-8176-03bb1596469c service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:55.025336 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1194/2381] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:53:54 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:53:58.237071 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ca6e7-afa4-4add-afcd-ea0992c41a7c None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:58.237630 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ca6e7-afa4-4add-afcd-ea0992c41a7c None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:58.237630 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ca6e7-afa4-4add-afcd-ea0992c41a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:58.245657 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:58.260428 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-681ca6e7-afa4-4add-afcd-ea0992c41a7c None None] MFA Rules not processed for user `a0e682af1a7f493c992509142b673932`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:58.260984 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-681ca6e7-afa4-4add-afcd-ea0992c41a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:58.268663 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1188/2382] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:53:58 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:54:05.020570 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=52 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:54:09.458038 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:54:11.962123 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484c2c02-4228-4c98-9bdd-b7cc54ccfd32 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:11.962615 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484c2c02-4228-4c98-9bdd-b7cc54ccfd32 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:11.962959 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484c2c02-4228-4c98-9bdd-b7cc54ccfd32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:12.024901 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-484c2c02-4228-4c98-9bdd-b7cc54ccfd32 None None] MFA Rules not processed for user `19d82c04f9994654adfb10f4bd0573c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:54:12.026726 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-484c2c02-4228-4c98-9bdd-b7cc54ccfd32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:54:12.096329 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1195/2383] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Jun 15 14:54:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:54:12.140798 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681ca6e7-afa4-4add-afcd-ea0992c41a7c None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:54:12.142261 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-681ca6e7-afa4-4add-afcd-ea0992c41a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:54:12.242418 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681ca6e7-afa4-4add-afcd-ea0992c41a7c None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:54:12.248179 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066cc699-a8b0-4f90-9739-fc463753fde8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04dc5832333498e96a7e02bbfb9484c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:54:12.248866 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066cc699-a8b0-4f90-9739-fc463753fde8 service swift] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:12.249148 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066cc699-a8b0-4f90-9739-fc463753fde8 service swift] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:12.249424 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066cc699-a8b0-4f90-9739-fc463753fde8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:12.259477 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066cc699-a8b0-4f90-9739-fc463753fde8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:54:12.261736 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066cc699-a8b0-4f90-9739-fc463753fde8 service swift] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:54:12.272515 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1189/2384] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Jun 15 14:54:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 14:54:13.237521 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dfbbba-5ed8-47ff-b931-89a74bfd188b None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:13.238074 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dfbbba-5ed8-47ff-b931-89a74bfd188b None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:13.238074 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dfbbba-5ed8-47ff-b931-89a74bfd188b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:13.259703 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-39dfbbba-5ed8-47ff-b931-89a74bfd188b None None] MFA Rules not processed for user `19d82c04f9994654adfb10f4bd0573c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:54:13.261444 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-39dfbbba-5ed8-47ff-b931-89a74bfd188b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:54:13.269263 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1196/2385] 10.222.0.42 () {58 vars in 967 bytes} [Wed Jun 15 14:54:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:54:13.287108 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066cc699-a8b0-4f90-9739-fc463753fde8 service swift] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:54:13.298709 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066cc699-a8b0-4f90-9739-fc463753fde8 service swift] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:54:13.304117 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e3d608-48af-4f52-bbb0-50cfc05f95a4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04dc5832333498e96a7e02bbfb9484c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:54:13.314367 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e3d608-48af-4f52-bbb0-50cfc05f95a4 service swift] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:13.314367 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e3d608-48af-4f52-bbb0-50cfc05f95a4 service swift] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:13.314367 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e3d608-48af-4f52-bbb0-50cfc05f95a4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:13.315443 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e3d608-48af-4f52-bbb0-50cfc05f95a4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:54:13.317106 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e3d608-48af-4f52-bbb0-50cfc05f95a4 service swift] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:54:13.326244 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1190/2386] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Jun 15 14:54:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 15 14:54:14.160460 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a51926c-43fa-4ec2-9b3e-fd78e2cf689a None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:14.160754 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a51926c-43fa-4ec2-9b3e-fd78e2cf689a None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:14.161040 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a51926c-43fa-4ec2-9b3e-fd78e2cf689a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:14.179812 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-8a51926c-43fa-4ec2-9b3e-fd78e2cf689a None None] MFA Rules not processed for user `19d82c04f9994654adfb10f4bd0573c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:54:14.181089 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-8a51926c-43fa-4ec2-9b3e-fd78e2cf689a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:54:14.188900 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1197/2387] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Jun 15 14:54:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:54:14.202397 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e3d608-48af-4f52-bbb0-50cfc05f95a4 service swift] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:54:14.214288 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e3d608-48af-4f52-bbb0-50cfc05f95a4 service swift] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:54:14.219822 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d937a6a5-0150-4b53-9175-37f894874461 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04dc5832333498e96a7e02bbfb9484c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:54:14.220536 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d937a6a5-0150-4b53-9175-37f894874461 service swift] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:14.220819 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d937a6a5-0150-4b53-9175-37f894874461 service swift] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:14.221097 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d937a6a5-0150-4b53-9175-37f894874461 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:14.230533 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d937a6a5-0150-4b53-9175-37f894874461 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:54:14.232231 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d937a6a5-0150-4b53-9175-37f894874461 service swift] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:54:14.241280 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1191/2388] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Jun 15 14:54:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 15 14:54:14.893404 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3946e9-7321-4fb7-8a95-7421201e59e8 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:14.893977 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3946e9-7321-4fb7-8a95-7421201e59e8 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:14.893977 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3946e9-7321-4fb7-8a95-7421201e59e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:14.913038 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-ba3946e9-7321-4fb7-8a95-7421201e59e8 None None] MFA Rules not processed for user `19d82c04f9994654adfb10f4bd0573c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:54:14.914323 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-ba3946e9-7321-4fb7-8a95-7421201e59e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:54:14.923048 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1198/2389] 10.222.0.42 () {58 vars in 967 bytes} [Wed Jun 15 14:54:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:54:14.936529 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d937a6a5-0150-4b53-9175-37f894874461 service swift] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:54:14.948013 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d937a6a5-0150-4b53-9175-37f894874461 service swift] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:54:14.953369 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c44ee28-765d-4741-9697-87de26430f5b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04dc5832333498e96a7e02bbfb9484c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:54:14.954044 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44ee28-765d-4741-9697-87de26430f5b service swift] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:14.954333 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44ee28-765d-4741-9697-87de26430f5b service swift] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:14.954614 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44ee28-765d-4741-9697-87de26430f5b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:14.963805 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c44ee28-765d-4741-9697-87de26430f5b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:54:14.965497 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c44ee28-765d-4741-9697-87de26430f5b service swift] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:54:14.975038 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1192/2390] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Jun 15 14:54:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 14:54:17.802239 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3946e9-7321-4fb7-8a95-7421201e59e8 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:54:17.871970 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3946e9-7321-4fb7-8a95-7421201e59e8 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:54:17.876698 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1472653c-3b5b-405f-a326-526a3e672002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:54:17.877572 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1472653c-3b5b-405f-a326-526a3e672002 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:17.877852 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1472653c-3b5b-405f-a326-526a3e672002 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:17.878130 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1472653c-3b5b-405f-a326-526a3e672002 admin admin] PATH_INFO: `/v3/users/be81f5426d684c95b21bf45ffbca87a2` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:17.879336 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1472653c-3b5b-405f-a326-526a3e672002 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be81f5426d684c95b21bf45ffbca87a2)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:54:17.881364 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1472653c-3b5b-405f-a326-526a3e672002 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:54:18.087595 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:54:18.089916 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-1472653c-3b5b-405f-a326-526a3e672002 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be81f5426d684c95b21bf45ffbca87a2'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:54:18.090619 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-1472653c-3b5b-405f-a326-526a3e672002 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be81f5426d684c95b21bf45ffbca87a2'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:54:18.091216 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-1472653c-3b5b-405f-a326-526a3e672002 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be81f5426d684c95b21bf45ffbca87a2'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:54:18.160727 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-1472653c-3b5b-405f-a326-526a3e672002 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be81f5426d684c95b21bf45ffbca87a2'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:54:18.175690 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-1472653c-3b5b-405f-a326-526a3e672002 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be81f5426d684c95b21bf45ffbca87a2'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:54:18.188881 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1199/2391] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:54:17 2022] DELETE /identity/v3/users/be81f5426d684c95b21bf45ffbca87a2 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:54:18.193571 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f90cd-2f58-4f62-b9d7-7545d6bf4fc8 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:18.194188 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f90cd-2f58-4f62-b9d7-7545d6bf4fc8 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:18.194188 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f90cd-2f58-4f62-b9d7-7545d6bf4fc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:18.211670 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-4d9f90cd-2f58-4f62-b9d7-7545d6bf4fc8 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:54:18.212913 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-4d9f90cd-2f58-4f62-b9d7-7545d6bf4fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:54:18.276253 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1193/2392] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:54:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:54:18.289090 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1472653c-3b5b-405f-a326-526a3e672002 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:54:18.290491 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-1472653c-3b5b-405f-a326-526a3e672002 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:54:18.350214 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1472653c-3b5b-405f-a326-526a3e672002 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:54:18.353691 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfc5e6b-79db-4a38-bced-7520db005f70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:54:18.354984 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc5e6b-79db-4a38-bced-7520db005f70 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:18.355256 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc5e6b-79db-4a38-bced-7520db005f70 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:18.355527 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc5e6b-79db-4a38-bced-7520db005f70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:18.364874 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfc5e6b-79db-4a38-bced-7520db005f70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:54:18.366302 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfc5e6b-79db-4a38-bced-7520db005f70 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:54:18.377140 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1200/2393] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:54:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:54:18.617323 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9f90cd-2f58-4f62-b9d7-7545d6bf4fc8 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:54:18.618749 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-4d9f90cd-2f58-4f62-b9d7-7545d6bf4fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:54:18.632843 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9f90cd-2f58-4f62-b9d7-7545d6bf4fc8 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:54:18.639458 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dbc920-54d6-452b-834c-9e51064468ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:54:18.640367 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dbc920-54d6-452b-834c-9e51064468ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:18.640651 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dbc920-54d6-452b-834c-9e51064468ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:18.640955 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dbc920-54d6-452b-834c-9e51064468ef admin admin] PATH_INFO: `/v3/projects/2addbe652f344f1583f0198fbd62864d` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:18.642245 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dbc920-54d6-452b-834c-9e51064468ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=2addbe652f344f1583f0198fbd62864d)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:54:18.644183 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dbc920-54d6-452b-834c-9e51064468ef admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:54:18.751094 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:54:18.768743 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-85dbc920-54d6-452b-834c-9e51064468ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2addbe652f344f1583f0198fbd62864d', 'request_id': 'req-85dbc920-54d6-452b-834c-9e51064468ef'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:54:18.829278 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-85dbc920-54d6-452b-834c-9e51064468ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2addbe652f344f1583f0198fbd62864d', 'request_id': 'req-85dbc920-54d6-452b-834c-9e51064468ef'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:54:18.835526 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-85dbc920-54d6-452b-834c-9e51064468ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2addbe652f344f1583f0198fbd62864d', 'request_id': 'req-85dbc920-54d6-452b-834c-9e51064468ef'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:54:18.846620 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-85dbc920-54d6-452b-834c-9e51064468ef admin admin] The token cache is being invalidate because project 2addbe652f344f1583f0198fbd62864d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:54:18.846904 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-85dbc920-54d6-452b-834c-9e51064468ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:54:18.847540 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-85dbc920-54d6-452b-834c-9e51064468ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:54:18.849131 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1194/2394] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:54:18 2022] DELETE /identity/v3/projects/2addbe652f344f1583f0198fbd62864d => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:54:18.854332 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ed4a9-142f-4882-8899-dc8974498bc6 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:18.854803 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ed4a9-142f-4882-8899-dc8974498bc6 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:18.854993 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ed4a9-142f-4882-8899-dc8974498bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:18.873180 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-f03ed4a9-142f-4882-8899-dc8974498bc6 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:54:18.874414 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-f03ed4a9-142f-4882-8899-dc8974498bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:54:18.935464 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1201/2395] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:54:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:54:18.943079 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47da7ab2-47ef-48a9-8ae4-0a8622036445 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:18.943391 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47da7ab2-47ef-48a9-8ae4-0a8622036445 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:18.943671 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47da7ab2-47ef-48a9-8ae4-0a8622036445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:18.962790 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-47da7ab2-47ef-48a9-8ae4-0a8622036445 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:54:18.964049 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-47da7ab2-47ef-48a9-8ae4-0a8622036445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:54:18.972120 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1195/2396] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:54:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:54:18.976284 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03ed4a9-142f-4882-8899-dc8974498bc6 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:54:18.986129 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03ed4a9-142f-4882-8899-dc8974498bc6 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:54:18.991187 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efecee91-3af9-4d67-8789-3f00a3f57c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:54:18.991835 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efecee91-3af9-4d67-8789-3f00a3f57c2f admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:18.992145 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efecee91-3af9-4d67-8789-3f00a3f57c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:18.992424 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efecee91-3af9-4d67-8789-3f00a3f57c2f admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:18.993187 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efecee91-3af9-4d67-8789-3f00a3f57c2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:54:18.993621 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efecee91-3af9-4d67-8789-3f00a3f57c2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:54:18.995829 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efecee91-3af9-4d67-8789-3f00a3f57c2f admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:54:19.007668 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1202/2397] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jun 15 14:54:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:54:19.018891 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47da7ab2-47ef-48a9-8ae4-0a8622036445 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:54:19.028274 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47da7ab2-47ef-48a9-8ae4-0a8622036445 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:54:19.033027 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80293085-b11a-4f21-8f9f-009cb48d3b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:54:19.033692 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80293085-b11a-4f21-8f9f-009cb48d3b8b admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:19.033971 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80293085-b11a-4f21-8f9f-009cb48d3b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:19.034249 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80293085-b11a-4f21-8f9f-009cb48d3b8b admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:19.034937 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80293085-b11a-4f21-8f9f-009cb48d3b8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:54:19.036884 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80293085-b11a-4f21-8f9f-009cb48d3b8b admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:54:19.128778 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1196/2398] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:54:19 2022] POST /identity/v3/projects => generated 381 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:54:19.132800 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efecee91-3af9-4d67-8789-3f00a3f57c2f admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:54:19.143738 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efecee91-3af9-4d67-8789-3f00a3f57c2f admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:54:19.192338 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:54:19.201170 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d249853-0719-4159-a63e-2d569fd41069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:54:19.201836 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d249853-0719-4159-a63e-2d569fd41069 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:19.202118 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d249853-0719-4159-a63e-2d569fd41069 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:19.206267 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d249853-0719-4159-a63e-2d569fd41069 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:19.206267 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d249853-0719-4159-a63e-2d569fd41069 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:54:19.206267 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d249853-0719-4159-a63e-2d569fd41069 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:54:19.287385 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1203/2399] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:54:19 2022] POST /identity/v3/users => generated 291 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:54:19.289432 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80293085-b11a-4f21-8f9f-009cb48d3b8b admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:54:19.301320 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80293085-b11a-4f21-8f9f-009cb48d3b8b admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:54:19.306079 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7baf45-321b-437b-9944-c9dd7f2ca4d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:54:19.306749 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7baf45-321b-437b-9944-c9dd7f2ca4d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:19.307757 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7baf45-321b-437b-9944-c9dd7f2ca4d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:19.308166 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7baf45-321b-437b-9944-c9dd7f2ca4d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:19.308900 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7baf45-321b-437b-9944-c9dd7f2ca4d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:54:19.310491 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7baf45-321b-437b-9944-c9dd7f2ca4d2 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:54:19.321180 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1197/2400] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:54:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:54:19.324994 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d249853-0719-4159-a63e-2d569fd41069 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:54:19.335383 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d249853-0719-4159-a63e-2d569fd41069 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:54:19.340343 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841dbbbd-a106-4eb2-a1ea-0c95f8f29ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:54:19.340998 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841dbbbd-a106-4eb2-a1ea-0c95f8f29ce2 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:19.341280 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841dbbbd-a106-4eb2-a1ea-0c95f8f29ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:19.341654 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841dbbbd-a106-4eb2-a1ea-0c95f8f29ce2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:19.342322 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841dbbbd-a106-4eb2-a1ea-0c95f8f29ce2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:54:19.343888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841dbbbd-a106-4eb2-a1ea-0c95f8f29ce2 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:54:19.355272 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1204/2401] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:54:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:54:19.359658 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7baf45-321b-437b-9944-c9dd7f2ca4d2 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:54:19.370375 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7baf45-321b-437b-9944-c9dd7f2ca4d2 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:54:19.375107 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31bbd1b-d7eb-41d0-b773-4bedb243b94b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:54:19.375980 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31bbd1b-d7eb-41d0-b773-4bedb243b94b admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:19.376343 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31bbd1b-d7eb-41d0-b773-4bedb243b94b admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:19.376627 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31bbd1b-d7eb-41d0-b773-4bedb243b94b admin admin] PATH_INFO: `/v3/projects/7a96a40d9feb40b8bbc143f1f9be7408/users/8a8c023cd5014daa993be2735b1c05ab/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:19.391610 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31bbd1b-d7eb-41d0-b773-4bedb243b94b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a96a40d9feb40b8bbc143f1f9be7408, user_id=8a8c023cd5014daa993be2735b1c05ab, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:54:19.394216 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31bbd1b-d7eb-41d0-b773-4bedb243b94b admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:54:19.457635 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1198/2402] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:54:19 2022] PUT /identity/v3/projects/7a96a40d9feb40b8bbc143f1f9be7408/users/8a8c023cd5014daa993be2735b1c05ab/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:54:19.462713 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84edddd-7397-44c2-a62a-a6a047921408 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:19.463003 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84edddd-7397-44c2-a62a-a6a047921408 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:19.463427 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84edddd-7397-44c2-a62a-a6a047921408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:19.481614 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-b84edddd-7397-44c2-a62a-a6a047921408 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:54:19.482830 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-b84edddd-7397-44c2-a62a-a6a047921408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:54:19.547414 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1205/2403] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:54:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:54:19.557604 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31bbd1b-d7eb-41d0-b773-4bedb243b94b admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:54:19.559544 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-f31bbd1b-d7eb-41d0-b773-4bedb243b94b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:54:19.618582 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31bbd1b-d7eb-41d0-b773-4bedb243b94b admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:54:19.622796 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5c7097-7202-4b44-b734-9b6fd7557e0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:54:19.623498 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5c7097-7202-4b44-b734-9b6fd7557e0e service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:19.623783 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5c7097-7202-4b44-b734-9b6fd7557e0e service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:19.624071 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5c7097-7202-4b44-b734-9b6fd7557e0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:19.637850 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5c7097-7202-4b44-b734-9b6fd7557e0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:54:19.639321 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5c7097-7202-4b44-b734-9b6fd7557e0e service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:54:19.649314 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1199/2404] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:54:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:54:25.634495 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69356310-5e80-4426-acac-706f06bdbd77 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:25.634993 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69356310-5e80-4426-acac-706f06bdbd77 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:25.635200 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69356310-5e80-4426-acac-706f06bdbd77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:25.682870 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-69356310-5e80-4426-acac-706f06bdbd77 None None] MFA Rules not processed for user `8a8c023cd5014daa993be2735b1c05ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:54:25.687665 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-69356310-5e80-4426-acac-706f06bdbd77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:54:25.810630 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1206/2405] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:54:25 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:54:25.814892 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5c7097-7202-4b44-b734-9b6fd7557e0e service neutron] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:54:25.825260 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5c7097-7202-4b44-b734-9b6fd7557e0e service neutron] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:54:25.830145 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca1724e-e3e1-4eb8-8a23-71e55f81f3c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:54:25.830884 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca1724e-e3e1-4eb8-8a23-71e55f81f3c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:25.831364 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca1724e-e3e1-4eb8-8a23-71e55f81f3c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:25.831634 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca1724e-e3e1-4eb8-8a23-71e55f81f3c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:25.832570 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca1724e-e3e1-4eb8-8a23-71e55f81f3c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:54:25.834472 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca1724e-e3e1-4eb8-8a23-71e55f81f3c3 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:54:25.938659 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1200/2406] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:54:25 2022] POST /identity/v3/projects => generated 383 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:54:25.940119 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69356310-5e80-4426-acac-706f06bdbd77 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:54:25.952731 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69356310-5e80-4426-acac-706f06bdbd77 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:54:26.023670 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e033930-a80d-4666-abce-ac550ff45fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:54:26.026115 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e033930-a80d-4666-abce-ac550ff45fb9 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:26.026115 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e033930-a80d-4666-abce-ac550ff45fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:26.026115 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e033930-a80d-4666-abce-ac550ff45fb9 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:26.028017 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e033930-a80d-4666-abce-ac550ff45fb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:54:26.032005 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e033930-a80d-4666-abce-ac550ff45fb9 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:54:26.045497 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:54:26.174715 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1207/2407] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:54:25 2022] POST /identity/v3/users => generated 298 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:54:26.179263 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca1724e-e3e1-4eb8-8a23-71e55f81f3c3 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:54:26.191251 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca1724e-e3e1-4eb8-8a23-71e55f81f3c3 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:54:26.201250 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc80810-7985-47ad-811a-c4f9585b0119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:54:26.202369 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc80810-7985-47ad-811a-c4f9585b0119 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:26.202971 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc80810-7985-47ad-811a-c4f9585b0119 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:26.203356 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc80810-7985-47ad-811a-c4f9585b0119 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:26.204430 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc80810-7985-47ad-811a-c4f9585b0119 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:54:26.207807 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc80810-7985-47ad-811a-c4f9585b0119 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:54:26.224030 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1201/2408] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:54:26 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:54:26.228322 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e033930-a80d-4666-abce-ac550ff45fb9 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:54:26.244645 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e033930-a80d-4666-abce-ac550ff45fb9 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:54:26.251471 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed17f6e5-a1bf-479d-8663-65d12f77f972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:54:26.252817 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed17f6e5-a1bf-479d-8663-65d12f77f972 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:26.253319 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed17f6e5-a1bf-479d-8663-65d12f77f972 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:26.253817 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed17f6e5-a1bf-479d-8663-65d12f77f972 admin admin] PATH_INFO: `/v3/projects/0560d2bbfd424594bd60ff80f99dcb14/users/668d27518c794b3cadbb76dc628e0683/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:26.280804 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed17f6e5-a1bf-479d-8663-65d12f77f972 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0560d2bbfd424594bd60ff80f99dcb14, user_id=668d27518c794b3cadbb76dc628e0683, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:54:26.285157 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed17f6e5-a1bf-479d-8663-65d12f77f972 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:54:26.364458 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1208/2409] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:54:26 2022] PUT /identity/v3/projects/0560d2bbfd424594bd60ff80f99dcb14/users/668d27518c794b3cadbb76dc628e0683/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:54:26.369280 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc80810-7985-47ad-811a-c4f9585b0119 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:54:26.380751 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc80810-7985-47ad-811a-c4f9585b0119 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:54:26.405590 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:54:26.455805 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a832a54d-276c-4535-b587-956b18c46d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:54:26.456569 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a832a54d-276c-4535-b587-956b18c46d0b admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:26.456857 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a832a54d-276c-4535-b587-956b18c46d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:26.457142 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a832a54d-276c-4535-b587-956b18c46d0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:26.457952 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a832a54d-276c-4535-b587-956b18c46d0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:54:26.459762 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a832a54d-276c-4535-b587-956b18c46d0b admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:54:26.473663 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1202/2410] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:54:26 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:54:26.482510 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed17f6e5-a1bf-479d-8663-65d12f77f972 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:54:26.493943 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed17f6e5-a1bf-479d-8663-65d12f77f972 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:54:26.499694 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5a0ef7-22ac-438e-adc2-d18412a889e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:54:26.500629 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a0ef7-22ac-438e-adc2-d18412a889e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:26.500912 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a0ef7-22ac-438e-adc2-d18412a889e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:26.501236 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a0ef7-22ac-438e-adc2-d18412a889e1 admin admin] PATH_INFO: `/v3/projects/0560d2bbfd424594bd60ff80f99dcb14/users/668d27518c794b3cadbb76dc628e0683/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:26.503315 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5a0ef7-22ac-438e-adc2-d18412a889e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0560d2bbfd424594bd60ff80f99dcb14, user_id=668d27518c794b3cadbb76dc628e0683, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:54:26.506006 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5a0ef7-22ac-438e-adc2-d18412a889e1 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:54:26.517624 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1209/2411] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:54:26 2022] PUT /identity/v3/projects/0560d2bbfd424594bd60ff80f99dcb14/users/668d27518c794b3cadbb76dc628e0683/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:54:34.432738 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bfc287-740d-4dba-8bb3-020f24affdda None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:34.433855 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bfc287-740d-4dba-8bb3-020f24affdda None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:34.433855 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bfc287-740d-4dba-8bb3-020f24affdda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:34.464009 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-33bfc287-740d-4dba-8bb3-020f24affdda None None] MFA Rules not processed for user `668d27518c794b3cadbb76dc628e0683`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:54:34.466275 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-33bfc287-740d-4dba-8bb3-020f24affdda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:54:34.564239 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1203/2412] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:54:34 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:54:34.578024 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5a0ef7-22ac-438e-adc2-d18412a889e1 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:54:34.579710 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-0e5a0ef7-22ac-438e-adc2-d18412a889e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:54:34.661144 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5a0ef7-22ac-438e-adc2-d18412a889e1 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:54:34.664825 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc59fb-6b76-4b94-9acb-0273c5abb46f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f18c57f27784694935bfb0bcea48a78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:54:34.665474 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc59fb-6b76-4b94-9acb-0273c5abb46f service glance] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:34.665756 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc59fb-6b76-4b94-9acb-0273c5abb46f service glance] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:34.666033 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc59fb-6b76-4b94-9acb-0273c5abb46f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:34.756756 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fc59fb-6b76-4b94-9acb-0273c5abb46f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:54:34.759212 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fc59fb-6b76-4b94-9acb-0273c5abb46f service glance] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:54:34.774258 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1210/2413] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:54:34 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:54:35.039998 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f856bed2-b00f-4b05-ac1a-b4ef0e9cb0bb None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:35.040322 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f856bed2-b00f-4b05-ac1a-b4ef0e9cb0bb None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:35.040606 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f856bed2-b00f-4b05-ac1a-b4ef0e9cb0bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:35.060433 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-f856bed2-b00f-4b05-ac1a-b4ef0e9cb0bb None None] MFA Rules not processed for user `19d82c04f9994654adfb10f4bd0573c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:54:35.061685 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-f856bed2-b00f-4b05-ac1a-b4ef0e9cb0bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:54:35.143891 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1204/2414] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Jun 15 14:54:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:54:35.160841 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc59fb-6b76-4b94-9acb-0273c5abb46f service glance] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:54:35.160841 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-10fc59fb-6b76-4b94-9acb-0273c5abb46f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:54:35.218186 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:54:35.238715 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc59fb-6b76-4b94-9acb-0273c5abb46f service glance] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:54:35.244214 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588fcc00-c94d-4db5-9b91-3c5e089f54bd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04dc5832333498e96a7e02bbfb9484c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:54:35.244985 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588fcc00-c94d-4db5-9b91-3c5e089f54bd service swift] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:35.245362 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588fcc00-c94d-4db5-9b91-3c5e089f54bd service swift] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:35.245765 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588fcc00-c94d-4db5-9b91-3c5e089f54bd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:35.256168 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588fcc00-c94d-4db5-9b91-3c5e089f54bd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:54:35.259101 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588fcc00-c94d-4db5-9b91-3c5e089f54bd service swift] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:54:35.267613 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1211/2415] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Jun 15 14:54:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 14:54:45.134709 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=79 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:54:45.726579 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=10 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:54:46.922923 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4ac52b-0a16-418a-be48-6b4b8d05ecad None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:46.923507 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4ac52b-0a16-418a-be48-6b4b8d05ecad None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:46.923507 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4ac52b-0a16-418a-be48-6b4b8d05ecad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:46.943741 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-ba4ac52b-0a16-418a-be48-6b4b8d05ecad None None] MFA Rules not processed for user `8a8c023cd5014daa993be2735b1c05ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:54:46.945052 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-ba4ac52b-0a16-418a-be48-6b4b8d05ecad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:54:46.953979 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1205/2416] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:54:46 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:54:54.470239 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75fe84b-7b14-4ff2-afcc-c4d043e6b0ad None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:54.470808 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75fe84b-7b14-4ff2-afcc-c4d043e6b0ad None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:54.470908 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75fe84b-7b14-4ff2-afcc-c4d043e6b0ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:54.494736 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-c75fe84b-7b14-4ff2-afcc-c4d043e6b0ad None None] MFA Rules not processed for user `19d82c04f9994654adfb10f4bd0573c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:54:54.496681 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c75fe84b-7b14-4ff2-afcc-c4d043e6b0ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:54:54.512668 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1212/2417] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Jun 15 14:54:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:54:54.530777 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4ac52b-0a16-418a-be48-6b4b8d05ecad None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:54:54.549198 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4ac52b-0a16-418a-be48-6b4b8d05ecad None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:54:54.555298 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260759a7-10de-4f83-a704-8c8b165254ab service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04dc5832333498e96a7e02bbfb9484c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:54:54.556070 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260759a7-10de-4f83-a704-8c8b165254ab service swift] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:54.556652 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260759a7-10de-4f83-a704-8c8b165254ab service swift] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:54.557343 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260759a7-10de-4f83-a704-8c8b165254ab service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:54.574738 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260759a7-10de-4f83-a704-8c8b165254ab service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:54:54.574738 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260759a7-10de-4f83-a704-8c8b165254ab service swift] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:54:54.585551 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1206/2418] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Jun 15 14:54:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 14:54:55.402590 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75fe84b-7b14-4ff2-afcc-c4d043e6b0ad None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:54:55.526786 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75fe84b-7b14-4ff2-afcc-c4d043e6b0ad None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:54:55.535884 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a5d582-7665-42af-9aea-38eb3e1dda91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:54:55.536690 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a5d582-7665-42af-9aea-38eb3e1dda91 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:55.537050 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a5d582-7665-42af-9aea-38eb3e1dda91 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:55.537441 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a5d582-7665-42af-9aea-38eb3e1dda91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:55.539802 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-64a5d582-7665-42af-9aea-38eb3e1dda91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:54:55.642652 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a5d582-7665-42af-9aea-38eb3e1dda91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:54:55.643999 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a5d582-7665-42af-9aea-38eb3e1dda91 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:54:55.657231 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1213/2419] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:54:55 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:55:04.579934 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=9 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:55:05.650591 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=53 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:55:09.089322 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260759a7-10de-4f83-a704-8c8b165254ab service swift] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:09.090957 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-260759a7-10de-4f83-a704-8c8b165254ab service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:09.104075 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260759a7-10de-4f83-a704-8c8b165254ab service swift] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:09.107956 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfd675f-25b1-4a9d-9430-2bc1a08b568f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:09.108664 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfd675f-25b1-4a9d-9430-2bc1a08b568f service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:09.108946 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfd675f-25b1-4a9d-9430-2bc1a08b568f service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:09.109227 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfd675f-25b1-4a9d-9430-2bc1a08b568f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:09.116492 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-cbfd675f-25b1-4a9d-9430-2bc1a08b568f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:09.200744 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfd675f-25b1-4a9d-9430-2bc1a08b568f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:09.202497 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfd675f-25b1-4a9d-9430-2bc1a08b568f service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:09.213353 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1207/2420] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:55:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:55:14.573211 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a5d582-7665-42af-9aea-38eb3e1dda91 service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:14.582618 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a5d582-7665-42af-9aea-38eb3e1dda91 service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:14.586258 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf0b0c8-fca5-4996-a9f0-c3100f1f80d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:14.586896 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf0b0c8-fca5-4996-a9f0-c3100f1f80d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:14.587180 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf0b0c8-fca5-4996-a9f0-c3100f1f80d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:14.587457 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf0b0c8-fca5-4996-a9f0-c3100f1f80d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:14.589328 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-dcf0b0c8-fca5-4996-a9f0-c3100f1f80d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:14.603846 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf0b0c8-fca5-4996-a9f0-c3100f1f80d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:14.605323 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf0b0c8-fca5-4996-a9f0-c3100f1f80d1 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:14.616879 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1214/2421] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:55:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:55:19.208484 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=34 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:55:24.611690 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=6 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:55:30.519889 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfd675f-25b1-4a9d-9430-2bc1a08b568f service neutron] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:30.521413 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-cbfd675f-25b1-4a9d-9430-2bc1a08b568f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:30.536111 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfd675f-25b1-4a9d-9430-2bc1a08b568f service neutron] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:30.542253 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4050ab32-3276-4743-85cd-55fa2fd56442 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0927c05c340c48e48b18754888aa9c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:30.542910 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4050ab32-3276-4743-85cd-55fa2fd56442 service nova] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:30.543201 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4050ab32-3276-4743-85cd-55fa2fd56442 service nova] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:30.543480 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4050ab32-3276-4743-85cd-55fa2fd56442 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:30.546138 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-4050ab32-3276-4743-85cd-55fa2fd56442 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:30.613976 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4050ab32-3276-4743-85cd-55fa2fd56442 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:30.615717 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4050ab32-3276-4743-85cd-55fa2fd56442 service nova] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:30.626407 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1208/2422] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:55:30 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:55:37.535954 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf0b0c8-fca5-4996-a9f0-c3100f1f80d1 service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:37.548126 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf0b0c8-fca5-4996-a9f0-c3100f1f80d1 service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:37.554036 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f126cdb5-0417-41ce-bade-19adfd151ebc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:37.554747 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f126cdb5-0417-41ce-bade-19adfd151ebc service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:37.555035 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f126cdb5-0417-41ce-bade-19adfd151ebc service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:37.555311 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f126cdb5-0417-41ce-bade-19adfd151ebc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:37.570097 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f126cdb5-0417-41ce-bade-19adfd151ebc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:37.572612 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f126cdb5-0417-41ce-bade-19adfd151ebc service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:37.586662 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1215/2423] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:55:37 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:55:38.590218 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3077da-ee30-4ad9-a685-798c65928aa0 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:38.590851 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3077da-ee30-4ad9-a685-798c65928aa0 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:38.590851 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3077da-ee30-4ad9-a685-798c65928aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:38.615585 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-0e3077da-ee30-4ad9-a685-798c65928aa0 None None] MFA Rules not processed for user `19d82c04f9994654adfb10f4bd0573c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:55:38.618181 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-0e3077da-ee30-4ad9-a685-798c65928aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:38.630426 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1209/2424] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Jun 15 14:55:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:55:38.668266 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f126cdb5-0417-41ce-bade-19adfd151ebc service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:38.682168 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f126cdb5-0417-41ce-bade-19adfd151ebc service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:38.688215 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891cf546-40fb-4691-bb0d-1db649381683 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04dc5832333498e96a7e02bbfb9484c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:38.688873 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891cf546-40fb-4691-bb0d-1db649381683 service swift] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:38.689156 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891cf546-40fb-4691-bb0d-1db649381683 service swift] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:38.689432 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891cf546-40fb-4691-bb0d-1db649381683 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:38.700315 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891cf546-40fb-4691-bb0d-1db649381683 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:38.700315 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891cf546-40fb-4691-bb0d-1db649381683 service swift] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:38.710436 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1216/2425] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Jun 15 14:55:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 14:55:40.509295 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3077da-ee30-4ad9-a685-798c65928aa0 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:40.523202 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3077da-ee30-4ad9-a685-798c65928aa0 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:40.529245 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c65929-b1ae-4046-9228-f6493b5863ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:40.529980 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c65929-b1ae-4046-9228-f6493b5863ed service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:40.530644 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c65929-b1ae-4046-9228-f6493b5863ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:40.531217 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c65929-b1ae-4046-9228-f6493b5863ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:40.533312 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-a8c65929-b1ae-4046-9228-f6493b5863ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:40.566626 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c65929-b1ae-4046-9228-f6493b5863ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:40.569253 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c65929-b1ae-4046-9228-f6493b5863ed service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:40.593459 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1210/2426] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:55:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:55:44.805421 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891cf546-40fb-4691-bb0d-1db649381683 service swift] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:44.807098 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-891cf546-40fb-4691-bb0d-1db649381683 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:44.850315 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891cf546-40fb-4691-bb0d-1db649381683 service swift] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:44.870186 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874aa8f9-cefa-4487-b7f3-4398e55bcb69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0927c05c340c48e48b18754888aa9c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:44.882904 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874aa8f9-cefa-4487-b7f3-4398e55bcb69 service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:44.883558 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874aa8f9-cefa-4487-b7f3-4398e55bcb69 service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:44.884003 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874aa8f9-cefa-4487-b7f3-4398e55bcb69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:44.887687 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-874aa8f9-cefa-4487-b7f3-4398e55bcb69 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:44.924380 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874aa8f9-cefa-4487-b7f3-4398e55bcb69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:44.929513 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874aa8f9-cefa-4487-b7f3-4398e55bcb69 service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:44.955685 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1217/2427] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:55:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:55:50.583724 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=37 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:55:54.942505 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=18 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:55:59.981757 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c65929-b1ae-4046-9228-f6493b5863ed service neutron] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:59.991059 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c65929-b1ae-4046-9228-f6493b5863ed service neutron] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:59.994614 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afe2900-ced2-4e9e-972a-1e679d736d20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:59.995243 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afe2900-ced2-4e9e-972a-1e679d736d20 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:59.995522 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afe2900-ced2-4e9e-972a-1e679d736d20 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:59.995796 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afe2900-ced2-4e9e-972a-1e679d736d20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:59.998383 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-8afe2900-ced2-4e9e-972a-1e679d736d20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:00.008258 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afe2900-ced2-4e9e-972a-1e679d736d20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:00.009672 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afe2900-ced2-4e9e-972a-1e679d736d20 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:00.017724 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1211/2428] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:55:59 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:56:00.371924 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3275b0c0-627d-4c5f-bb16-7d0ac7650ebb None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:00.372458 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3275b0c0-627d-4c5f-bb16-7d0ac7650ebb None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:00.372564 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3275b0c0-627d-4c5f-bb16-7d0ac7650ebb None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:00.373882 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1218/2429] 10.222.0.42 () {54 vars in 921 bytes} [Wed Jun 15 14:56:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:56:00.380971 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c2f27-e913-418b-bcb7-70ad71d5e231 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:00.381262 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c2f27-e913-418b-bcb7-70ad71d5e231 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:00.381541 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c2f27-e913-418b-bcb7-70ad71d5e231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:00.420421 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-b62c2f27-e913-418b-bcb7-70ad71d5e231 None None] MFA Rules not processed for user `0927c05c340c48e48b18754888aa9c35`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:00.421711 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-b62c2f27-e913-418b-bcb7-70ad71d5e231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:00.432662 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1212/2430] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Jun 15 14:56:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:56:00.448653 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3275b0c0-627d-4c5f-bb16-7d0ac7650ebb None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:00.461219 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3275b0c0-627d-4c5f-bb16-7d0ac7650ebb None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:00.464701 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32af3c35-ff3d-49f7-b927-c5375418c924 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f18c57f27784694935bfb0bcea48a78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:00.465340 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af3c35-ff3d-49f7-b927-c5375418c924 service glance] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:00.465622 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af3c35-ff3d-49f7-b927-c5375418c924 service glance] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:00.465896 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af3c35-ff3d-49f7-b927-c5375418c924 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:00.477445 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32af3c35-ff3d-49f7-b927-c5375418c924 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:00.478881 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32af3c35-ff3d-49f7-b927-c5375418c924 service glance] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:00.490892 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1219/2431] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:56:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:56:00.712172 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9286eb-9f62-4d8e-b506-1b0aa54068e5 None None] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:00.712381 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9286eb-9f62-4d8e-b506-1b0aa54068e5 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:00.712662 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9286eb-9f62-4d8e-b506-1b0aa54068e5 None None] PATH_INFO: `/` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:00.714065 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1213/2432] 10.222.0.42 () {54 vars in 921 bytes} [Wed Jun 15 14:56:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:56:00.718954 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8038b3d-8c36-48a5-89da-f9e3d920df0c None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:00.719327 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8038b3d-8c36-48a5-89da-f9e3d920df0c None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:00.721438 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8038b3d-8c36-48a5-89da-f9e3d920df0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:00.738496 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-e8038b3d-8c36-48a5-89da-f9e3d920df0c None None] MFA Rules not processed for user `c739b61c8a8b453e979bda720bafebb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:00.739704 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e8038b3d-8c36-48a5-89da-f9e3d920df0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:00.746266 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1220/2433] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Jun 15 14:56:00 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:56:00.759382 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9286eb-9f62-4d8e-b506-1b0aa54068e5 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:00.771769 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9286eb-9f62-4d8e-b506-1b0aa54068e5 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:00.775369 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b11a28-277c-4d68-861d-215a580c7746 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:00.776002 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b11a28-277c-4d68-861d-215a580c7746 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:00.776316 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b11a28-277c-4d68-861d-215a580c7746 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:00.776592 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b11a28-277c-4d68-861d-215a580c7746 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:00.783770 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b11a28-277c-4d68-861d-215a580c7746 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:00.785231 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b11a28-277c-4d68-861d-215a580c7746 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:00.793009 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1214/2434] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:56:00 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:56:00.877103 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87898bad-4296-45d4-a529-db775de7a8ab None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:00.877315 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87898bad-4296-45d4-a529-db775de7a8ab None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:00.877576 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87898bad-4296-45d4-a529-db775de7a8ab None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:00.878983 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1221/2435] 10.222.0.42 () {54 vars in 921 bytes} [Wed Jun 15 14:56:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:56:00.884897 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f670d0-5988-44ac-aae5-5f7f0e4d42dc None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:00.885198 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f670d0-5988-44ac-aae5-5f7f0e4d42dc None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:00.885482 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f670d0-5988-44ac-aae5-5f7f0e4d42dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:00.904791 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-21f670d0-5988-44ac-aae5-5f7f0e4d42dc None None] MFA Rules not processed for user `0927c05c340c48e48b18754888aa9c35`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:00.906320 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-21f670d0-5988-44ac-aae5-5f7f0e4d42dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:00.915642 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1215/2436] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Jun 15 14:56:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:56:00.926421 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87898bad-4296-45d4-a529-db775de7a8ab None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:00.938197 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87898bad-4296-45d4-a529-db775de7a8ab None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:00.942178 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b271c13-fd2c-418a-847d-a86f471e1a1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:00.942832 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b271c13-fd2c-418a-847d-a86f471e1a1e service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:00.943116 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b271c13-fd2c-418a-847d-a86f471e1a1e service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:00.943393 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b271c13-fd2c-418a-847d-a86f471e1a1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:00.954828 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b271c13-fd2c-418a-847d-a86f471e1a1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:00.956301 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b271c13-fd2c-418a-847d-a86f471e1a1e service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:00.968136 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1222/2437] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:56:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:56:01.034756 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f670d0-5988-44ac-aae5-5f7f0e4d42dc None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:01.037060 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-21f670d0-5988-44ac-aae5-5f7f0e4d42dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:01.116422 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f670d0-5988-44ac-aae5-5f7f0e4d42dc None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:01.122178 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63518c4b-a13c-4ae1-8a1d-5d93da2c6a7b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bb6671c19464424a6e9cd4519ac93e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:01.122835 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63518c4b-a13c-4ae1-8a1d-5d93da2c6a7b service placement] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:01.123120 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63518c4b-a13c-4ae1-8a1d-5d93da2c6a7b service placement] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:01.123400 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63518c4b-a13c-4ae1-8a1d-5d93da2c6a7b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:01.126329 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-63518c4b-a13c-4ae1-8a1d-5d93da2c6a7b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:01.139643 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63518c4b-a13c-4ae1-8a1d-5d93da2c6a7b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:01.141766 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63518c4b-a13c-4ae1-8a1d-5d93da2c6a7b service placement] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:01.153886 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1216/2438] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:56:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:56:05.705032 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d43fe99-77c0-4563-98c9-23e16f4a7e0e None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:05.705547 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d43fe99-77c0-4563-98c9-23e16f4a7e0e None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:05.705701 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d43fe99-77c0-4563-98c9-23e16f4a7e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:05.723912 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-5d43fe99-77c0-4563-98c9-23e16f4a7e0e None None] MFA Rules not processed for user `19d82c04f9994654adfb10f4bd0573c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:05.725225 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-5d43fe99-77c0-4563-98c9-23e16f4a7e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:05.732755 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1223/2439] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Jun 15 14:56:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:56:05.745037 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63518c4b-a13c-4ae1-8a1d-5d93da2c6a7b service placement] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:05.757544 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63518c4b-a13c-4ae1-8a1d-5d93da2c6a7b service placement] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:05.763076 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8876ccaf-d6c7-40c8-86d9-83fd6806f124 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04dc5832333498e96a7e02bbfb9484c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:05.763717 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8876ccaf-d6c7-40c8-86d9-83fd6806f124 service swift] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:05.764212 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8876ccaf-d6c7-40c8-86d9-83fd6806f124 service swift] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:05.764358 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8876ccaf-d6c7-40c8-86d9-83fd6806f124 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:05.773304 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8876ccaf-d6c7-40c8-86d9-83fd6806f124 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:05.774866 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8876ccaf-d6c7-40c8-86d9-83fd6806f124 service swift] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:05.783659 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1217/2440] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Jun 15 14:56:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 14:56:13.971860 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d43fe99-77c0-4563-98c9-23e16f4a7e0e None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:13.991750 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d43fe99-77c0-4563-98c9-23e16f4a7e0e None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:13.997703 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8526816c-fe46-4d44-9030-1fdf7b7edd6d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0927c05c340c48e48b18754888aa9c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:13.998361 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8526816c-fe46-4d44-9030-1fdf7b7edd6d service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:13.998669 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8526816c-fe46-4d44-9030-1fdf7b7edd6d service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:13.998948 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8526816c-fe46-4d44-9030-1fdf7b7edd6d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:14.000833 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-8526816c-fe46-4d44-9030-1fdf7b7edd6d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:14.063158 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8526816c-fe46-4d44-9030-1fdf7b7edd6d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:14.064830 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8526816c-fe46-4d44-9030-1fdf7b7edd6d service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:14.074138 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1224/2441] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:56:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:56:15.779764 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=62 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:56:24.069894 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=46 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:56:29.843751 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8876ccaf-d6c7-40c8-86d9-83fd6806f124 service swift] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:29.856834 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8876ccaf-d6c7-40c8-86d9-83fd6806f124 service swift] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:29.862502 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b20716-3e16-4e26-99aa-88de35aebb53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0927c05c340c48e48b18754888aa9c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:29.863134 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b20716-3e16-4e26-99aa-88de35aebb53 service nova] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:29.863415 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b20716-3e16-4e26-99aa-88de35aebb53 service nova] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:29.863697 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b20716-3e16-4e26-99aa-88de35aebb53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:29.866472 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-a8b20716-3e16-4e26-99aa-88de35aebb53 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:29.878633 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b20716-3e16-4e26-99aa-88de35aebb53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:29.880830 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b20716-3e16-4e26-99aa-88de35aebb53 service nova] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:29.891935 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1218/2442] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:56:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:56:32.814378 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c48bbf4-199a-40bd-827f-afc208766a1c None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:32.814959 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c48bbf4-199a-40bd-827f-afc208766a1c None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:32.814959 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c48bbf4-199a-40bd-827f-afc208766a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:32.832856 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-7c48bbf4-199a-40bd-827f-afc208766a1c None None] MFA Rules not processed for user `ee9df934d1cd461abbf9c3cd90a8fdda`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:32.834087 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-7c48bbf4-199a-40bd-827f-afc208766a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:32.843128 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1225/2443] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:56:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:56:35.830412 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d0ff54-7186-41ea-b03a-22adecf34108 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:35.830940 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d0ff54-7186-41ea-b03a-22adecf34108 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:35.831042 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d0ff54-7186-41ea-b03a-22adecf34108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:35.851108 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-d7d0ff54-7186-41ea-b03a-22adecf34108 None None] MFA Rules not processed for user `19d82c04f9994654adfb10f4bd0573c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:35.852349 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-d7d0ff54-7186-41ea-b03a-22adecf34108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:35.860141 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1219/2444] 10.222.0.42 () {58 vars in 967 bytes} [Wed Jun 15 14:56:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:56:35.874864 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c48bbf4-199a-40bd-827f-afc208766a1c None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:35.889063 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c48bbf4-199a-40bd-827f-afc208766a1c None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:35.894502 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d33143b-d2bb-4c04-9b39-fd8c5daab780 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04dc5832333498e96a7e02bbfb9484c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:35.895171 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d33143b-d2bb-4c04-9b39-fd8c5daab780 service swift] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:35.895510 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d33143b-d2bb-4c04-9b39-fd8c5daab780 service swift] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:35.896000 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d33143b-d2bb-4c04-9b39-fd8c5daab780 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:35.905684 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d33143b-d2bb-4c04-9b39-fd8c5daab780 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:35.907310 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d33143b-d2bb-4c04-9b39-fd8c5daab780 service swift] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:35.917103 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1226/2445] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Jun 15 14:56:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 15 14:56:45.844562 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=9 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:56:45.913292 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=9 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:57.855289 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d0ff54-7186-41ea-b03a-22adecf34108 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:57.870341 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d0ff54-7186-41ea-b03a-22adecf34108 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:57.876145 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35463bd4-d68b-4656-85da-b155f82b2934 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0927c05c340c48e48b18754888aa9c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:57.876795 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35463bd4-d68b-4656-85da-b155f82b2934 service nova] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:57.877076 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35463bd4-d68b-4656-85da-b155f82b2934 service nova] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:57.877350 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35463bd4-d68b-4656-85da-b155f82b2934 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:57.879485 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-35463bd4-d68b-4656-85da-b155f82b2934 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:57.942757 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35463bd4-d68b-4656-85da-b155f82b2934 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:57.944505 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35463bd4-d68b-4656-85da-b155f82b2934 service nova] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:57.955607 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1220/2446] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:57:57 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:58:07.949788 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=28 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:58:14.146961 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d33143b-d2bb-4c04-9b39-fd8c5daab780 service swift] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:14.158382 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-7d33143b-d2bb-4c04-9b39-fd8c5daab780 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:14.169234 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d33143b-d2bb-4c04-9b39-fd8c5daab780 service swift] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:14.175978 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e635cb5-6508-4cf7-908d-30084a54c069 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bb6671c19464424a6e9cd4519ac93e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:14.178274 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e635cb5-6508-4cf7-908d-30084a54c069 service placement] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:14.178808 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e635cb5-6508-4cf7-908d-30084a54c069 service placement] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:14.179304 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e635cb5-6508-4cf7-908d-30084a54c069 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:14.181526 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-8e635cb5-6508-4cf7-908d-30084a54c069 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:14.203449 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e635cb5-6508-4cf7-908d-30084a54c069 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:14.205807 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e635cb5-6508-4cf7-908d-30084a54c069 service placement] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:14.220172 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1227/2447] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:58:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:58:15.896533 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c48b0-6bec-4587-980b-50823860a3e1 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:15.897048 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c48b0-6bec-4587-980b-50823860a3e1 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:15.902751 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c48b0-6bec-4587-980b-50823860a3e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:15.917868 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-283c48b0-6bec-4587-980b-50823860a3e1 None None] MFA Rules not processed for user `2dcc02eb960e45b99587438aa3696f40`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:58:15.919036 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-283c48b0-6bec-4587-980b-50823860a3e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:15.926394 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1221/2448] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:58:15 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:58:16.334172 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e635cb5-6508-4cf7-908d-30084a54c069 service placement] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:16.343754 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e635cb5-6508-4cf7-908d-30084a54c069 service placement] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:16.350134 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604a04b4-1f6e-49b3-9538-4c467ba1e286 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:16.350805 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a04b4-1f6e-49b3-9538-4c467ba1e286 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:16.351238 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a04b4-1f6e-49b3-9538-4c467ba1e286 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:16.351513 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a04b4-1f6e-49b3-9538-4c467ba1e286 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:16.353330 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-604a04b4-1f6e-49b3-9538-4c467ba1e286 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:16.366824 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604a04b4-1f6e-49b3-9538-4c467ba1e286 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:16.368276 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604a04b4-1f6e-49b3-9538-4c467ba1e286 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:16.376503 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1228/2449] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:58:16 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:58:25.911147 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:58:26.373065 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=12 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:58:38.209104 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283c48b0-6bec-4587-980b-50823860a3e1 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:38.220537 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283c48b0-6bec-4587-980b-50823860a3e1 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:38.225508 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fb1b4c-3d2d-4685-8bcc-a446d47026a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:38.226185 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb1b4c-3d2d-4685-8bcc-a446d47026a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:38.226474 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb1b4c-3d2d-4685-8bcc-a446d47026a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:38.226756 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb1b4c-3d2d-4685-8bcc-a446d47026a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:38.229033 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-25fb1b4c-3d2d-4685-8bcc-a446d47026a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:38.315945 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fb1b4c-3d2d-4685-8bcc-a446d47026a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:38.318769 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fb1b4c-3d2d-4685-8bcc-a446d47026a5 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:38.334102 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1222/2450] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:58:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:58:43.530917 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604a04b4-1f6e-49b3-9538-4c467ba1e286 service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:43.546267 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604a04b4-1f6e-49b3-9538-4c467ba1e286 service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:43.553868 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c637e8-82f2-476a-b361-f7ae02a00e1a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bb6671c19464424a6e9cd4519ac93e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:43.558725 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c637e8-82f2-476a-b361-f7ae02a00e1a service placement] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:43.558725 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c637e8-82f2-476a-b361-f7ae02a00e1a service placement] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:43.558725 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c637e8-82f2-476a-b361-f7ae02a00e1a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:43.566635 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-d9c637e8-82f2-476a-b361-f7ae02a00e1a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:43.573895 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c637e8-82f2-476a-b361-f7ae02a00e1a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:43.575595 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c637e8-82f2-476a-b361-f7ae02a00e1a service placement] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:43.587549 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1229/2451] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:58:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:58:43.727485 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fb1b4c-3d2d-4685-8bcc-a446d47026a5 service neutron] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:43.734944 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-25fb1b4c-3d2d-4685-8bcc-a446d47026a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:43.743007 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fb1b4c-3d2d-4685-8bcc-a446d47026a5 service neutron] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:43.753404 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d724c2e-1c97-4801-8770-f051e71203e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:43.754441 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d724c2e-1c97-4801-8770-f051e71203e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:43.754798 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d724c2e-1c97-4801-8770-f051e71203e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:43.755142 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d724c2e-1c97-4801-8770-f051e71203e1 admin admin] PATH_INFO: `/v3/users/ee9df934d1cd461abbf9c3cd90a8fdda` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:43.756632 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d724c2e-1c97-4801-8770-f051e71203e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee9df934d1cd461abbf9c3cd90a8fdda)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:43.758802 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d724c2e-1c97-4801-8770-f051e71203e1 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:44.047189 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-9d724c2e-1c97-4801-8770-f051e71203e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee9df934d1cd461abbf9c3cd90a8fdda'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:58:44.047867 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-9d724c2e-1c97-4801-8770-f051e71203e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee9df934d1cd461abbf9c3cd90a8fdda'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:58:44.058116 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-9d724c2e-1c97-4801-8770-f051e71203e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee9df934d1cd461abbf9c3cd90a8fdda'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:58:44.059647 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-9d724c2e-1c97-4801-8770-f051e71203e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee9df934d1cd461abbf9c3cd90a8fdda'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:58:44.152410 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-9d724c2e-1c97-4801-8770-f051e71203e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee9df934d1cd461abbf9c3cd90a8fdda'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:58:44.173248 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1223/2452] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:58:43 2022] DELETE /identity/v3/users/ee9df934d1cd461abbf9c3cd90a8fdda => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:58:44.439971 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c637e8-82f2-476a-b361-f7ae02a00e1a service placement] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:44.454748 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c637e8-82f2-476a-b361-f7ae02a00e1a service placement] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:44.530354 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc978d1d-588a-4421-8f6e-3693c4fff041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:44.531221 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc978d1d-588a-4421-8f6e-3693c4fff041 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:44.532033 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc978d1d-588a-4421-8f6e-3693c4fff041 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:44.532033 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc978d1d-588a-4421-8f6e-3693c4fff041 admin admin] PATH_INFO: `/v3/projects/b356eba3587e481aa04f99bcb2194b11` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:44.533001 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc978d1d-588a-4421-8f6e-3693c4fff041 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b356eba3587e481aa04f99bcb2194b11)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:44.535764 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc978d1d-588a-4421-8f6e-3693c4fff041 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:44.689663 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-dc978d1d-588a-4421-8f6e-3693c4fff041 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b356eba3587e481aa04f99bcb2194b11', 'request_id': 'req-dc978d1d-588a-4421-8f6e-3693c4fff041'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:58:44.702391 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-dc978d1d-588a-4421-8f6e-3693c4fff041 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b356eba3587e481aa04f99bcb2194b11', 'request_id': 'req-dc978d1d-588a-4421-8f6e-3693c4fff041'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:58:44.759300 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-dc978d1d-588a-4421-8f6e-3693c4fff041 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b356eba3587e481aa04f99bcb2194b11', 'request_id': 'req-dc978d1d-588a-4421-8f6e-3693c4fff041'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:58:44.772099 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-dc978d1d-588a-4421-8f6e-3693c4fff041 admin admin] The token cache is being invalidate because project b356eba3587e481aa04f99bcb2194b11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:58:44.772490 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-dc978d1d-588a-4421-8f6e-3693c4fff041 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:58:44.773550 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-dc978d1d-588a-4421-8f6e-3693c4fff041 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:58:44.777074 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1230/2453] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:58:44 2022] DELETE /identity/v3/projects/b356eba3587e481aa04f99bcb2194b11 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:58:44.782651 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6457baf1-1ed7-4cd1-873e-83915b19e58d None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:44.783195 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6457baf1-1ed7-4cd1-873e-83915b19e58d None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:44.783985 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6457baf1-1ed7-4cd1-873e-83915b19e58d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:44.803810 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-6457baf1-1ed7-4cd1-873e-83915b19e58d None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:58:44.805821 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-6457baf1-1ed7-4cd1-873e-83915b19e58d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:44.875483 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1224/2454] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:58:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:58:44.884482 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b18fc-52f4-4a4b-844b-d1a3b1f329e3 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:44.884773 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b18fc-52f4-4a4b-844b-d1a3b1f329e3 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:44.885051 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b18fc-52f4-4a4b-844b-d1a3b1f329e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:44.904795 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-b67b18fc-52f4-4a4b-844b-d1a3b1f329e3 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:58:44.906006 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-b67b18fc-52f4-4a4b-844b-d1a3b1f329e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:44.914287 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1231/2455] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:58:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:58:44.918690 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6457baf1-1ed7-4cd1-873e-83915b19e58d None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:44.933740 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6457baf1-1ed7-4cd1-873e-83915b19e58d None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:44.936411 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d70bad4-835f-4991-963d-a826ced52dc8 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:44.936782 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d70bad4-835f-4991-963d-a826ced52dc8 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:44.937061 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d70bad4-835f-4991-963d-a826ced52dc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:44.939281 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b092fd5c-cf64-4ef7-89d1-1a8598267413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:44.939777 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b092fd5c-cf64-4ef7-89d1-1a8598267413 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:44.940869 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b092fd5c-cf64-4ef7-89d1-1a8598267413 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:44.940869 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b092fd5c-cf64-4ef7-89d1-1a8598267413 admin admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:44.941649 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b092fd5c-cf64-4ef7-89d1-1a8598267413 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:58:44.941649 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b092fd5c-cf64-4ef7-89d1-1a8598267413 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:44.943530 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b092fd5c-cf64-4ef7-89d1-1a8598267413 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:44.945285 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:58:44.957424 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1225/2456] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jun 15 14:58:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:58:44.957986 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-0d70bad4-835f-4991-963d-a826ced52dc8 None None] MFA Rules not processed for user `0650eb65117a4e27908c4355bfc46c3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:58:44.959300 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-0d70bad4-835f-4991-963d-a826ced52dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:44.962021 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b092fd5c-cf64-4ef7-89d1-1a8598267413 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:44.972181 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b092fd5c-cf64-4ef7-89d1-1a8598267413 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:44.977339 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f2a844-c7cc-4320-a258-47da1d899c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:44.978524 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f2a844-c7cc-4320-a258-47da1d899c93 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:44.978814 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f2a844-c7cc-4320-a258-47da1d899c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:44.979101 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f2a844-c7cc-4320-a258-47da1d899c93 admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:44.980231 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f2a844-c7cc-4320-a258-47da1d899c93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:44.984307 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f2a844-c7cc-4320-a258-47da1d899c93 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:45.028477 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1232/2457] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:58:44 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:58:45.092596 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1226/2458] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:58:44 2022] POST /identity/v3/projects => generated 385 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:58:45.095967 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d70bad4-835f-4991-963d-a826ced52dc8 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:45.111651 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d70bad4-835f-4991-963d-a826ced52dc8 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:45.128756 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:58:45.191595 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5d75ba-c644-4eeb-8816-10dd8f218cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:45.192340 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d75ba-c644-4eeb-8816-10dd8f218cd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:45.192656 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d75ba-c644-4eeb-8816-10dd8f218cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:45.192938 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d75ba-c644-4eeb-8816-10dd8f218cd6 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:45.193673 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5d75ba-c644-4eeb-8816-10dd8f218cd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:45.195627 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5d75ba-c644-4eeb-8816-10dd8f218cd6 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:45.364028 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1233/2459] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:58:45 2022] POST /identity/v3/users => generated 293 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:58:45.368423 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f2a844-c7cc-4320-a258-47da1d899c93 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:45.376699 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f2a844-c7cc-4320-a258-47da1d899c93 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:45.382378 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924a6549-dc85-4903-a4c0-eefc00e68322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:45.383194 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924a6549-dc85-4903-a4c0-eefc00e68322 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:45.383476 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924a6549-dc85-4903-a4c0-eefc00e68322 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:45.383755 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924a6549-dc85-4903-a4c0-eefc00e68322 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:45.384510 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924a6549-dc85-4903-a4c0-eefc00e68322 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:45.386189 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924a6549-dc85-4903-a4c0-eefc00e68322 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:45.397107 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1227/2460] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:58:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:58:45.402511 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5d75ba-c644-4eeb-8816-10dd8f218cd6 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:45.414748 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5d75ba-c644-4eeb-8816-10dd8f218cd6 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:45.419561 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7588a66f-6bf6-4f1f-a8e1-49ddf455cb32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:45.420310 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7588a66f-6bf6-4f1f-a8e1-49ddf455cb32 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:45.420623 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7588a66f-6bf6-4f1f-a8e1-49ddf455cb32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:45.420904 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7588a66f-6bf6-4f1f-a8e1-49ddf455cb32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:45.421687 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7588a66f-6bf6-4f1f-a8e1-49ddf455cb32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:45.423336 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7588a66f-6bf6-4f1f-a8e1-49ddf455cb32 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:45.438018 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1234/2461] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:58:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:58:45.439262 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924a6549-dc85-4903-a4c0-eefc00e68322 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:45.449390 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924a6549-dc85-4903-a4c0-eefc00e68322 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:45.459914 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76591e23-8fc5-4f9c-8888-5fd4b40f8ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:45.460942 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76591e23-8fc5-4f9c-8888-5fd4b40f8ebc admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:45.461122 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76591e23-8fc5-4f9c-8888-5fd4b40f8ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:45.461410 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76591e23-8fc5-4f9c-8888-5fd4b40f8ebc admin admin] PATH_INFO: `/v3/projects/053765c8fc8342c6b155fdd11264fc24/users/cbae2a0ce784498fb526fc0b21d00cf5/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:45.479522 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76591e23-8fc5-4f9c-8888-5fd4b40f8ebc admin admin] RBAC: Authorizing `identity:create_grant(project_id=053765c8fc8342c6b155fdd11264fc24, user_id=cbae2a0ce784498fb526fc0b21d00cf5, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:45.482838 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76591e23-8fc5-4f9c-8888-5fd4b40f8ebc admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:45.559528 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1228/2462] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:58:45 2022] PUT /identity/v3/projects/053765c8fc8342c6b155fdd11264fc24/users/cbae2a0ce784498fb526fc0b21d00cf5/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:58:45.570847 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e922ff-5bd8-4578-9de8-87ca741703f0 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:45.571540 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e922ff-5bd8-4578-9de8-87ca741703f0 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:45.571540 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e922ff-5bd8-4578-9de8-87ca741703f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:45.603658 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-c8e922ff-5bd8-4578-9de8-87ca741703f0 None None] MFA Rules not processed for user `cbae2a0ce784498fb526fc0b21d00cf5`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:58:45.604927 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c8e922ff-5bd8-4578-9de8-87ca741703f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:45.677351 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1235/2463] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:58:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:58:45.687873 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76591e23-8fc5-4f9c-8888-5fd4b40f8ebc admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:45.689574 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-76591e23-8fc5-4f9c-8888-5fd4b40f8ebc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:45.769818 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76591e23-8fc5-4f9c-8888-5fd4b40f8ebc admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:45.780554 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0776fc-f8be-449b-8237-ed361dec417c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0927c05c340c48e48b18754888aa9c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:45.781216 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0776fc-f8be-449b-8237-ed361dec417c service nova] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:45.781496 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0776fc-f8be-449b-8237-ed361dec417c service nova] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:45.781771 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0776fc-f8be-449b-8237-ed361dec417c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:45.792188 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0776fc-f8be-449b-8237-ed361dec417c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:45.793784 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0776fc-f8be-449b-8237-ed361dec417c service nova] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:45.804125 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1229/2464] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:58:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:58:55.673628 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=59,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:58:55.800157 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=65,keystone:INSERT=2 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:58:58.337195 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e922ff-5bd8-4578-9de8-87ca741703f0 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:58.422814 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e922ff-5bd8-4578-9de8-87ca741703f0 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:58.428936 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e1d388-1262-42e8-aa3a-b13217ae6963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:58.429853 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e1d388-1262-42e8-aa3a-b13217ae6963 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:58.430243 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e1d388-1262-42e8-aa3a-b13217ae6963 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:58.430524 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e1d388-1262-42e8-aa3a-b13217ae6963 admin admin] PATH_INFO: `/v3/users/cbae2a0ce784498fb526fc0b21d00cf5` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:58.431787 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e1d388-1262-42e8-aa3a-b13217ae6963 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbae2a0ce784498fb526fc0b21d00cf5)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:58.433747 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e1d388-1262-42e8-aa3a-b13217ae6963 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:58.683422 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-05e1d388-1262-42e8-aa3a-b13217ae6963 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbae2a0ce784498fb526fc0b21d00cf5'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:58:58.684214 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-05e1d388-1262-42e8-aa3a-b13217ae6963 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbae2a0ce784498fb526fc0b21d00cf5'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:58:58.685329 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-05e1d388-1262-42e8-aa3a-b13217ae6963 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbae2a0ce784498fb526fc0b21d00cf5'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:58:58.765617 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-05e1d388-1262-42e8-aa3a-b13217ae6963 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbae2a0ce784498fb526fc0b21d00cf5'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:58:58.789021 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-05e1d388-1262-42e8-aa3a-b13217ae6963 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbae2a0ce784498fb526fc0b21d00cf5'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:58:58.805343 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1236/2465] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:58:58 2022] DELETE /identity/v3/users/cbae2a0ce784498fb526fc0b21d00cf5 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:58:58.810190 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b0d190-56ef-48bc-8479-0317e23202f3 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:58.810748 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b0d190-56ef-48bc-8479-0317e23202f3 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:58.810850 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b0d190-56ef-48bc-8479-0317e23202f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:58.831721 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-47b0d190-56ef-48bc-8479-0317e23202f3 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:58:58.833054 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-47b0d190-56ef-48bc-8479-0317e23202f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:58.903832 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1230/2466] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:58:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:58:58.915315 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e1d388-1262-42e8-aa3a-b13217ae6963 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:58.918863 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-05e1d388-1262-42e8-aa3a-b13217ae6963 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:58.985054 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e1d388-1262-42e8-aa3a-b13217ae6963 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:58.989406 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eed8a1-b19c-40c3-be2b-0f534cba5ff1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:58.990086 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eed8a1-b19c-40c3-be2b-0f534cba5ff1 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:58.990376 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eed8a1-b19c-40c3-be2b-0f534cba5ff1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:58.990669 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eed8a1-b19c-40c3-be2b-0f534cba5ff1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:59.000972 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eed8a1-b19c-40c3-be2b-0f534cba5ff1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:59.003214 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eed8a1-b19c-40c3-be2b-0f534cba5ff1 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:59.014537 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1237/2467] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:58:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:58:59.428843 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b0d190-56ef-48bc-8479-0317e23202f3 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:59.430321 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-47b0d190-56ef-48bc-8479-0317e23202f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:59.456544 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b0d190-56ef-48bc-8479-0317e23202f3 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:59.463828 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9228a229-79bc-4d65-9a9c-a166ae4359dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:59.464908 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9228a229-79bc-4d65-9a9c-a166ae4359dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:59.465193 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9228a229-79bc-4d65-9a9c-a166ae4359dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:59.465469 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9228a229-79bc-4d65-9a9c-a166ae4359dd admin admin] PATH_INFO: `/v3/projects/053765c8fc8342c6b155fdd11264fc24` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:59.466836 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9228a229-79bc-4d65-9a9c-a166ae4359dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=053765c8fc8342c6b155fdd11264fc24)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:59.468932 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9228a229-79bc-4d65-9a9c-a166ae4359dd admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:59.579623 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-9228a229-79bc-4d65-9a9c-a166ae4359dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '053765c8fc8342c6b155fdd11264fc24', 'request_id': 'req-9228a229-79bc-4d65-9a9c-a166ae4359dd'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:58:59.670017 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-9228a229-79bc-4d65-9a9c-a166ae4359dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '053765c8fc8342c6b155fdd11264fc24', 'request_id': 'req-9228a229-79bc-4d65-9a9c-a166ae4359dd'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:58:59.678222 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-9228a229-79bc-4d65-9a9c-a166ae4359dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '053765c8fc8342c6b155fdd11264fc24', 'request_id': 'req-9228a229-79bc-4d65-9a9c-a166ae4359dd'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:58:59.694293 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-9228a229-79bc-4d65-9a9c-a166ae4359dd admin admin] The token cache is being invalidate because project 053765c8fc8342c6b155fdd11264fc24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:58:59.694594 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-9228a229-79bc-4d65-9a9c-a166ae4359dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:58:59.695406 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-9228a229-79bc-4d65-9a9c-a166ae4359dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:58:59.697833 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1231/2468] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:58:59 2022] DELETE /identity/v3/projects/053765c8fc8342c6b155fdd11264fc24 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:58:59.703423 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f7043b-1a8a-4a4a-8c04-3d6a70e6cc0b None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:59.703941 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f7043b-1a8a-4a4a-8c04-3d6a70e6cc0b None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:59.705046 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f7043b-1a8a-4a4a-8c04-3d6a70e6cc0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:59.730470 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-94f7043b-1a8a-4a4a-8c04-3d6a70e6cc0b None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:58:59.732338 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-94f7043b-1a8a-4a4a-8c04-3d6a70e6cc0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:59.790755 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:58:59.879334 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1238/2469] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:58:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:58:59.885038 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2f7d2-c5f3-4a9d-a7e5-c68b71a545f3 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:59.885921 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2f7d2-c5f3-4a9d-a7e5-c68b71a545f3 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:59.886413 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2f7d2-c5f3-4a9d-a7e5-c68b71a545f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:59.905954 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-a9b2f7d2-c5f3-4a9d-a7e5-c68b71a545f3 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:58:59.907546 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-a9b2f7d2-c5f3-4a9d-a7e5-c68b71a545f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:59.918715 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1232/2470] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:58:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:58:59.923136 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f7043b-1a8a-4a4a-8c04-3d6a70e6cc0b None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:59.937329 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f7043b-1a8a-4a4a-8c04-3d6a70e6cc0b None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:59.945853 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48bc960-7a22-4568-a46f-22cd0dc7399e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:59.946937 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48bc960-7a22-4568-a46f-22cd0dc7399e admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:59.947428 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48bc960-7a22-4568-a46f-22cd0dc7399e admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:59.947952 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48bc960-7a22-4568-a46f-22cd0dc7399e admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:59.949359 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48bc960-7a22-4568-a46f-22cd0dc7399e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:58:59.950841 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48bc960-7a22-4568-a46f-22cd0dc7399e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:59.952791 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48bc960-7a22-4568-a46f-22cd0dc7399e admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:59.972306 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1239/2471] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jun 15 14:58:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:58:59.976174 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b2f7d2-c5f3-4a9d-a7e5-c68b71a545f3 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:59.990706 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b2f7d2-c5f3-4a9d-a7e5-c68b71a545f3 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:59.999939 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5daa59-eb09-46cb-b17d-b65cb2645c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:00.000734 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5daa59-eb09-46cb-b17d-b65cb2645c2b admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:00.001087 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5daa59-eb09-46cb-b17d-b65cb2645c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:00.001440 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5daa59-eb09-46cb-b17d-b65cb2645c2b admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:00.003556 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5daa59-eb09-46cb-b17d-b65cb2645c2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:00.006039 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5daa59-eb09-46cb-b17d-b65cb2645c2b admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:00.109356 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1233/2472] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:58:59 2022] POST /identity/v3/projects => generated 373 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:59:00.114253 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48bc960-7a22-4568-a46f-22cd0dc7399e admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:00.125001 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48bc960-7a22-4568-a46f-22cd0dc7399e admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:00.205075 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68dde59-ee43-4b1c-99d0-33e8dad79277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:00.206348 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68dde59-ee43-4b1c-99d0-33e8dad79277 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:00.206576 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68dde59-ee43-4b1c-99d0-33e8dad79277 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:00.208182 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68dde59-ee43-4b1c-99d0-33e8dad79277 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:00.208182 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68dde59-ee43-4b1c-99d0-33e8dad79277 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:00.212515 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68dde59-ee43-4b1c-99d0-33e8dad79277 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:00.330317 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1240/2473] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:59:00 2022] POST /identity/v3/users => generated 287 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:59:00.340126 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5daa59-eb09-46cb-b17d-b65cb2645c2b admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:00.355885 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5daa59-eb09-46cb-b17d-b65cb2645c2b admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:00.360681 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d88d10-835a-4924-9d9a-671f68a51e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:00.361566 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d88d10-835a-4924-9d9a-671f68a51e4c admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:00.362368 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d88d10-835a-4924-9d9a-671f68a51e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:00.362815 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d88d10-835a-4924-9d9a-671f68a51e4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:00.363801 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d88d10-835a-4924-9d9a-671f68a51e4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:00.365475 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d88d10-835a-4924-9d9a-671f68a51e4c admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:00.382027 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1234/2474] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:59:00 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:59:00.384037 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68dde59-ee43-4b1c-99d0-33e8dad79277 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:00.397635 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68dde59-ee43-4b1c-99d0-33e8dad79277 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:00.402493 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ca41a3-3023-4d56-9848-0267ee33b30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:00.403197 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca41a3-3023-4d56-9848-0267ee33b30e admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:00.403477 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca41a3-3023-4d56-9848-0267ee33b30e admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:00.403748 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca41a3-3023-4d56-9848-0267ee33b30e admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:00.404508 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ca41a3-3023-4d56-9848-0267ee33b30e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:00.406091 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ca41a3-3023-4d56-9848-0267ee33b30e admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:00.422184 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1241/2475] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:59:00 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:59:00.423396 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d88d10-835a-4924-9d9a-671f68a51e4c admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:00.434763 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d88d10-835a-4924-9d9a-671f68a51e4c admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:00.439647 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5486e753-140f-4baa-8821-30ca3b0f017d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:00.442886 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5486e753-140f-4baa-8821-30ca3b0f017d admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:00.442886 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5486e753-140f-4baa-8821-30ca3b0f017d admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:00.442886 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5486e753-140f-4baa-8821-30ca3b0f017d admin admin] PATH_INFO: `/v3/projects/df4d0ffecc7f40b4a34f51c714d96fa5/users/c1e9c64746444cd3ba00d33d5898a8e0/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:00.462223 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5486e753-140f-4baa-8821-30ca3b0f017d admin admin] RBAC: Authorizing `identity:create_grant(project_id=df4d0ffecc7f40b4a34f51c714d96fa5, user_id=c1e9c64746444cd3ba00d33d5898a8e0, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:00.464868 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5486e753-140f-4baa-8821-30ca3b0f017d admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:00.542915 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1235/2476] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:59:00 2022] PUT /identity/v3/projects/df4d0ffecc7f40b4a34f51c714d96fa5/users/c1e9c64746444cd3ba00d33d5898a8e0/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:59:00.554591 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43024d16-96e3-46ae-a6df-a63e1f90dbf2 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:00.555044 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43024d16-96e3-46ae-a6df-a63e1f90dbf2 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:00.555858 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43024d16-96e3-46ae-a6df-a63e1f90dbf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:00.586592 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-43024d16-96e3-46ae-a6df-a63e1f90dbf2 None None] MFA Rules not processed for user `c1e9c64746444cd3ba00d33d5898a8e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:59:00.587881 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-43024d16-96e3-46ae-a6df-a63e1f90dbf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:00.670655 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1242/2477] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:59:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:59:00.684699 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5486e753-140f-4baa-8821-30ca3b0f017d admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:00.686465 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-5486e753-140f-4baa-8821-30ca3b0f017d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:00.711436 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:59:00.776524 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5486e753-140f-4baa-8821-30ca3b0f017d admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:00.782913 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b52adc8-f33b-48c0-9d8d-8e6a404fb408 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0927c05c340c48e48b18754888aa9c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:00.783874 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b52adc8-f33b-48c0-9d8d-8e6a404fb408 service nova] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:00.784135 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b52adc8-f33b-48c0-9d8d-8e6a404fb408 service nova] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:00.784297 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b52adc8-f33b-48c0-9d8d-8e6a404fb408 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:00.793471 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b52adc8-f33b-48c0-9d8d-8e6a404fb408 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:00.795120 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b52adc8-f33b-48c0-9d8d-8e6a404fb408 service nova] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:00.805220 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1236/2478] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:59:00 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:59:05.374123 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43024d16-96e3-46ae-a6df-a63e1f90dbf2 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:05.392079 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:59:05.442947 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43024d16-96e3-46ae-a6df-a63e1f90dbf2 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:05.447631 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cdcfaf-7964-4fe4-a378-597024760210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:05.448537 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cdcfaf-7964-4fe4-a378-597024760210 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:05.448824 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cdcfaf-7964-4fe4-a378-597024760210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:05.449111 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cdcfaf-7964-4fe4-a378-597024760210 admin admin] PATH_INFO: `/v3/users/c1e9c64746444cd3ba00d33d5898a8e0` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:05.450326 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cdcfaf-7964-4fe4-a378-597024760210 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1e9c64746444cd3ba00d33d5898a8e0)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:05.452300 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cdcfaf-7964-4fe4-a378-597024760210 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:05.657448 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-59cdcfaf-7964-4fe4-a378-597024760210 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1e9c64746444cd3ba00d33d5898a8e0'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:05.658110 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-59cdcfaf-7964-4fe4-a378-597024760210 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1e9c64746444cd3ba00d33d5898a8e0'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:05.658740 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-59cdcfaf-7964-4fe4-a378-597024760210 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1e9c64746444cd3ba00d33d5898a8e0'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:05.734591 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-59cdcfaf-7964-4fe4-a378-597024760210 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1e9c64746444cd3ba00d33d5898a8e0'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:05.747346 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-59cdcfaf-7964-4fe4-a378-597024760210 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1e9c64746444cd3ba00d33d5898a8e0'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:05.760841 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1243/2479] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:59:05 2022] DELETE /identity/v3/users/c1e9c64746444cd3ba00d33d5898a8e0 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:59:05.765562 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32306c51-92b9-4f30-8528-bcfff698f2e5 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:05.766068 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32306c51-92b9-4f30-8528-bcfff698f2e5 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:05.766170 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32306c51-92b9-4f30-8528-bcfff698f2e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:05.784062 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-32306c51-92b9-4f30-8528-bcfff698f2e5 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:59:05.785385 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-32306c51-92b9-4f30-8528-bcfff698f2e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:05.847793 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1237/2480] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:59:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:59:05.857617 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cdcfaf-7964-4fe4-a378-597024760210 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:05.859031 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-59cdcfaf-7964-4fe4-a378-597024760210 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:05.918551 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cdcfaf-7964-4fe4-a378-597024760210 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:05.922318 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de932b9e-071c-4783-a2a9-a4993d30a208 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:05.922969 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de932b9e-071c-4783-a2a9-a4993d30a208 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:05.923251 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de932b9e-071c-4783-a2a9-a4993d30a208 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:05.923531 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de932b9e-071c-4783-a2a9-a4993d30a208 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:05.933168 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de932b9e-071c-4783-a2a9-a4993d30a208 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:05.934611 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de932b9e-071c-4783-a2a9-a4993d30a208 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:05.944957 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1244/2481] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:59:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:59:06.503881 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32306c51-92b9-4f30-8528-bcfff698f2e5 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:06.505396 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-32306c51-92b9-4f30-8528-bcfff698f2e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:06.524809 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32306c51-92b9-4f30-8528-bcfff698f2e5 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:06.531357 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572ff9f4-f6df-4e29-8602-afad36d134b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:06.532271 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ff9f4-f6df-4e29-8602-afad36d134b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:06.532656 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ff9f4-f6df-4e29-8602-afad36d134b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:06.532819 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ff9f4-f6df-4e29-8602-afad36d134b2 admin admin] PATH_INFO: `/v3/projects/df4d0ffecc7f40b4a34f51c714d96fa5` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:06.534954 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572ff9f4-f6df-4e29-8602-afad36d134b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df4d0ffecc7f40b4a34f51c714d96fa5)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:06.536925 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572ff9f4-f6df-4e29-8602-afad36d134b2 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:06.648382 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-572ff9f4-f6df-4e29-8602-afad36d134b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df4d0ffecc7f40b4a34f51c714d96fa5', 'request_id': 'req-572ff9f4-f6df-4e29-8602-afad36d134b2'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:06.755958 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-572ff9f4-f6df-4e29-8602-afad36d134b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df4d0ffecc7f40b4a34f51c714d96fa5', 'request_id': 'req-572ff9f4-f6df-4e29-8602-afad36d134b2'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:06.762903 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-572ff9f4-f6df-4e29-8602-afad36d134b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df4d0ffecc7f40b4a34f51c714d96fa5', 'request_id': 'req-572ff9f4-f6df-4e29-8602-afad36d134b2'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:06.776140 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-572ff9f4-f6df-4e29-8602-afad36d134b2 admin admin] The token cache is being invalidate because project df4d0ffecc7f40b4a34f51c714d96fa5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:59:06.776982 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-572ff9f4-f6df-4e29-8602-afad36d134b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:06.777530 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-572ff9f4-f6df-4e29-8602-afad36d134b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:06.779842 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1238/2482] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:59:06 2022] DELETE /identity/v3/projects/df4d0ffecc7f40b4a34f51c714d96fa5 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:59:06.785426 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c35cb36-4d54-42df-8f3e-f10ff90e4beb None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:06.786088 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c35cb36-4d54-42df-8f3e-f10ff90e4beb None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:06.786088 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c35cb36-4d54-42df-8f3e-f10ff90e4beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:06.803996 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-8c35cb36-4d54-42df-8f3e-f10ff90e4beb None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:59:06.805307 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-8c35cb36-4d54-42df-8f3e-f10ff90e4beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:06.840358 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:59:06.868564 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1245/2483] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:59:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:59:06.879720 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769520e2-7f9f-4b0f-a8b6-ea4a05da2a17 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:06.880050 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769520e2-7f9f-4b0f-a8b6-ea4a05da2a17 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:06.880345 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769520e2-7f9f-4b0f-a8b6-ea4a05da2a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:06.900630 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-769520e2-7f9f-4b0f-a8b6-ea4a05da2a17 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:59:06.901758 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-769520e2-7f9f-4b0f-a8b6-ea4a05da2a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:06.910513 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1239/2484] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:59:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:59:06.915084 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c35cb36-4d54-42df-8f3e-f10ff90e4beb None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:06.924945 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c35cb36-4d54-42df-8f3e-f10ff90e4beb None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:06.930005 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bb4352-c9e7-479d-a925-c05e8fcc11ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:06.930678 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bb4352-c9e7-479d-a925-c05e8fcc11ae admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:06.930964 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bb4352-c9e7-479d-a925-c05e8fcc11ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:06.936434 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bb4352-c9e7-479d-a925-c05e8fcc11ae admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:06.936434 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bb4352-c9e7-479d-a925-c05e8fcc11ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:59:06.936434 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bb4352-c9e7-479d-a925-c05e8fcc11ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:06.936434 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bb4352-c9e7-479d-a925-c05e8fcc11ae admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:06.946382 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1246/2485] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jun 15 14:59:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:59:06.950832 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769520e2-7f9f-4b0f-a8b6-ea4a05da2a17 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:06.961191 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769520e2-7f9f-4b0f-a8b6-ea4a05da2a17 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:06.966012 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa6d98e-441b-41d3-ba2b-539827dd5328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:06.966672 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6d98e-441b-41d3-ba2b-539827dd5328 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:06.966958 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6d98e-441b-41d3-ba2b-539827dd5328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:06.967237 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6d98e-441b-41d3-ba2b-539827dd5328 admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:06.967971 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa6d98e-441b-41d3-ba2b-539827dd5328 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:06.969977 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa6d98e-441b-41d3-ba2b-539827dd5328 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:07.106060 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:59:07.112889 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1240/2486] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:59:06 2022] POST /identity/v3/projects => generated 387 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:59:07.117828 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bb4352-c9e7-479d-a925-c05e8fcc11ae admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:07.129319 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bb4352-c9e7-479d-a925-c05e8fcc11ae admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:07.188171 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1133ca7b-bcc5-440f-b8a2-9c91b8b03f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:07.188908 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1133ca7b-bcc5-440f-b8a2-9c91b8b03f0e admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:07.189217 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1133ca7b-bcc5-440f-b8a2-9c91b8b03f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:07.189469 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1133ca7b-bcc5-440f-b8a2-9c91b8b03f0e admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:07.190653 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1133ca7b-bcc5-440f-b8a2-9c91b8b03f0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:07.192601 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1133ca7b-bcc5-440f-b8a2-9c91b8b03f0e admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:07.289400 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1247/2487] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:59:07 2022] POST /identity/v3/users => generated 294 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:59:07.293982 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa6d98e-441b-41d3-ba2b-539827dd5328 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:07.304870 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa6d98e-441b-41d3-ba2b-539827dd5328 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:07.309671 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d8ef3d-a0c0-432f-8cde-baa581d46e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:07.310377 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d8ef3d-a0c0-432f-8cde-baa581d46e2f admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:07.310658 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d8ef3d-a0c0-432f-8cde-baa581d46e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:07.310934 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d8ef3d-a0c0-432f-8cde-baa581d46e2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:07.311707 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d8ef3d-a0c0-432f-8cde-baa581d46e2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:07.313353 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d8ef3d-a0c0-432f-8cde-baa581d46e2f admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:07.324648 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1241/2488] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:59:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:59:07.328728 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1133ca7b-bcc5-440f-b8a2-9c91b8b03f0e admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:07.340282 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1133ca7b-bcc5-440f-b8a2-9c91b8b03f0e admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:07.345016 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a604f1-3c71-41a7-b949-703473943f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:07.345677 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a604f1-3c71-41a7-b949-703473943f58 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:07.346075 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a604f1-3c71-41a7-b949-703473943f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:07.346324 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a604f1-3c71-41a7-b949-703473943f58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:07.347065 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a604f1-3c71-41a7-b949-703473943f58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:07.348741 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a604f1-3c71-41a7-b949-703473943f58 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:07.359528 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1248/2489] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:59:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:59:07.364162 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d8ef3d-a0c0-432f-8cde-baa581d46e2f admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:07.376076 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d8ef3d-a0c0-432f-8cde-baa581d46e2f admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:07.381390 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044c43fd-f32e-4754-a5de-3e103c3c89c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:07.382290 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044c43fd-f32e-4754-a5de-3e103c3c89c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:07.382579 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044c43fd-f32e-4754-a5de-3e103c3c89c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:07.382854 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044c43fd-f32e-4754-a5de-3e103c3c89c4 admin admin] PATH_INFO: `/v3/projects/dcc34dcadd0d4c2c9d571c81a4714cbc/users/ea9787c9ccea4015bb3b4ca88ec887e5/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:07.404711 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044c43fd-f32e-4754-a5de-3e103c3c89c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcc34dcadd0d4c2c9d571c81a4714cbc, user_id=ea9787c9ccea4015bb3b4ca88ec887e5, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:07.407389 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044c43fd-f32e-4754-a5de-3e103c3c89c4 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:07.520923 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1242/2490] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:59:07 2022] PUT /identity/v3/projects/dcc34dcadd0d4c2c9d571c81a4714cbc/users/ea9787c9ccea4015bb3b4ca88ec887e5/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:59:07.532071 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab384b28-80fa-4eab-8280-b1fd66c8bda4 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:07.532592 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab384b28-80fa-4eab-8280-b1fd66c8bda4 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:07.532752 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab384b28-80fa-4eab-8280-b1fd66c8bda4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:07.585802 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-ab384b28-80fa-4eab-8280-b1fd66c8bda4 None None] MFA Rules not processed for user `ea9787c9ccea4015bb3b4ca88ec887e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:59:07.587193 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-ab384b28-80fa-4eab-8280-b1fd66c8bda4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:07.672144 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1249/2491] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:59:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:59:07.675204 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044c43fd-f32e-4754-a5de-3e103c3c89c4 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:07.760514 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044c43fd-f32e-4754-a5de-3e103c3c89c4 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:07.765400 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdb8445-7ec1-4521-9bcd-59ed84b64fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:07.766098 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdb8445-7ec1-4521-9bcd-59ed84b64fdb admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:07.766382 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdb8445-7ec1-4521-9bcd-59ed84b64fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:07.766760 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdb8445-7ec1-4521-9bcd-59ed84b64fdb admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:07.768194 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdb8445-7ec1-4521-9bcd-59ed84b64fdb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:07.770141 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdb8445-7ec1-4521-9bcd-59ed84b64fdb admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:07.858229 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1243/2492] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:59:07 2022] POST /identity/v3/projects => generated 387 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:59:07.863189 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab384b28-80fa-4eab-8280-b1fd66c8bda4 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:07.880317 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab384b28-80fa-4eab-8280-b1fd66c8bda4 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:07.917292 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:59:07.946870 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c9411c-2d94-4098-87b1-49c71b71b240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:07.947563 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c9411c-2d94-4098-87b1-49c71b71b240 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:07.947849 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c9411c-2d94-4098-87b1-49c71b71b240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:07.948159 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c9411c-2d94-4098-87b1-49c71b71b240 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:07.948867 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c9411c-2d94-4098-87b1-49c71b71b240 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:07.950961 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c9411c-2d94-4098-87b1-49c71b71b240 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:08.056532 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1250/2493] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:59:07 2022] POST /identity/v3/users => generated 300 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:59:08.060841 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdb8445-7ec1-4521-9bcd-59ed84b64fdb admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:08.072746 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdb8445-7ec1-4521-9bcd-59ed84b64fdb admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:08.077631 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d6f59a-a2bd-4cd5-afbc-43670eb70b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:08.078394 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d6f59a-a2bd-4cd5-afbc-43670eb70b1e admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:08.078678 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d6f59a-a2bd-4cd5-afbc-43670eb70b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:08.078995 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d6f59a-a2bd-4cd5-afbc-43670eb70b1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:08.079828 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d6f59a-a2bd-4cd5-afbc-43670eb70b1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:08.081874 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d6f59a-a2bd-4cd5-afbc-43670eb70b1e admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:08.093839 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1244/2494] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:59:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:59:08.103554 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c9411c-2d94-4098-87b1-49c71b71b240 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:08.110983 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c9411c-2d94-4098-87b1-49c71b71b240 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:08.117724 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa0a83-8050-4523-a9e3-cf505cdc1508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:08.118548 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa0a83-8050-4523-a9e3-cf505cdc1508 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:08.118830 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa0a83-8050-4523-a9e3-cf505cdc1508 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:08.119105 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa0a83-8050-4523-a9e3-cf505cdc1508 admin admin] PATH_INFO: `/v3/projects/0d2e4a8f03334314817d6343922f999e/users/3554e294cb3447f5b6873e3594c5b686/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:08.136342 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caa0a83-8050-4523-a9e3-cf505cdc1508 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d2e4a8f03334314817d6343922f999e, user_id=3554e294cb3447f5b6873e3594c5b686, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:08.139031 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caa0a83-8050-4523-a9e3-cf505cdc1508 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:08.283026 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1251/2495] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:59:08 2022] PUT /identity/v3/projects/0d2e4a8f03334314817d6343922f999e/users/3554e294cb3447f5b6873e3594c5b686/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:59:08.288316 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d6f59a-a2bd-4cd5-afbc-43670eb70b1e admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:08.299793 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d6f59a-a2bd-4cd5-afbc-43670eb70b1e admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:08.365205 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e59261a-b27d-4c3b-800d-82b7dc1d5eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:08.365946 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e59261a-b27d-4c3b-800d-82b7dc1d5eb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:08.366292 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e59261a-b27d-4c3b-800d-82b7dc1d5eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:08.366635 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e59261a-b27d-4c3b-800d-82b7dc1d5eb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:08.367435 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e59261a-b27d-4c3b-800d-82b7dc1d5eb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:08.369344 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e59261a-b27d-4c3b-800d-82b7dc1d5eb4 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:08.380790 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1245/2496] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:59:08 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:59:08.385637 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa0a83-8050-4523-a9e3-cf505cdc1508 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:08.397888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa0a83-8050-4523-a9e3-cf505cdc1508 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:08.403256 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58443ce0-a934-4947-b191-aced03388114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:08.404195 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58443ce0-a934-4947-b191-aced03388114 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:08.404484 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58443ce0-a934-4947-b191-aced03388114 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:08.404782 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58443ce0-a934-4947-b191-aced03388114 admin admin] PATH_INFO: `/v3/projects/0d2e4a8f03334314817d6343922f999e/users/3554e294cb3447f5b6873e3594c5b686/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:08.406811 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58443ce0-a934-4947-b191-aced03388114 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d2e4a8f03334314817d6343922f999e, user_id=3554e294cb3447f5b6873e3594c5b686, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:08.410698 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58443ce0-a934-4947-b191-aced03388114 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:08.422691 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1252/2497] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:59:08 2022] PUT /identity/v3/projects/0d2e4a8f03334314817d6343922f999e/users/3554e294cb3447f5b6873e3594c5b686/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:59:08.433652 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e32aea-077e-4ea7-b6c6-0e0c78b93f72 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:08.433947 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e32aea-077e-4ea7-b6c6-0e0c78b93f72 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:08.434231 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e32aea-077e-4ea7-b6c6-0e0c78b93f72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:08.464615 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-70e32aea-077e-4ea7-b6c6-0e0c78b93f72 None None] MFA Rules not processed for user `3554e294cb3447f5b6873e3594c5b686`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:59:08.466139 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-70e32aea-077e-4ea7-b6c6-0e0c78b93f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:08.494403 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:59:08.557439 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1246/2498] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:59:08 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:59:08.571976 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58443ce0-a934-4947-b191-aced03388114 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:08.574411 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-58443ce0-a934-4947-b191-aced03388114 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:08.653486 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58443ce0-a934-4947-b191-aced03388114 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:08.657566 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aecdd7-457a-4f7d-acb8-094b09e2d702 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:08.658192 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aecdd7-457a-4f7d-acb8-094b09e2d702 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:08.659226 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aecdd7-457a-4f7d-acb8-094b09e2d702 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:08.659602 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aecdd7-457a-4f7d-acb8-094b09e2d702 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:08.671154 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aecdd7-457a-4f7d-acb8-094b09e2d702 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:08.672698 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aecdd7-457a-4f7d-acb8-094b09e2d702 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:08.684159 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1253/2499] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:59:08 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:59:08.776328 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e32aea-077e-4ea7-b6c6-0e0c78b93f72 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:08.843502 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e32aea-077e-4ea7-b6c6-0e0c78b93f72 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:08.849006 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8e2193-d1ff-4121-ab66-248ac1bdb375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:08.849610 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e2193-d1ff-4121-ab66-248ac1bdb375 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:08.849891 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e2193-d1ff-4121-ab66-248ac1bdb375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:08.850169 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e2193-d1ff-4121-ab66-248ac1bdb375 admin admin] PATH_INFO: `/v3/users/ea9787c9ccea4015bb3b4ca88ec887e5` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:08.851473 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8e2193-d1ff-4121-ab66-248ac1bdb375 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea9787c9ccea4015bb3b4ca88ec887e5)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:08.853411 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8e2193-d1ff-4121-ab66-248ac1bdb375 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:09.058654 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-ab8e2193-d1ff-4121-ab66-248ac1bdb375 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea9787c9ccea4015bb3b4ca88ec887e5'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:09.060375 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-ab8e2193-d1ff-4121-ab66-248ac1bdb375 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea9787c9ccea4015bb3b4ca88ec887e5'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:09.074583 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-ab8e2193-d1ff-4121-ab66-248ac1bdb375 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea9787c9ccea4015bb3b4ca88ec887e5'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:09.079148 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-ab8e2193-d1ff-4121-ab66-248ac1bdb375 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea9787c9ccea4015bb3b4ca88ec887e5'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:09.198692 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-ab8e2193-d1ff-4121-ab66-248ac1bdb375 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea9787c9ccea4015bb3b4ca88ec887e5'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:09.226782 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1247/2500] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:59:08 2022] DELETE /identity/v3/users/ea9787c9ccea4015bb3b4ca88ec887e5 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:59:09.231518 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8237acf9-8f2d-41e3-8839-acc7f3cf8436 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:09.232059 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8237acf9-8f2d-41e3-8839-acc7f3cf8436 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:09.232177 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8237acf9-8f2d-41e3-8839-acc7f3cf8436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:09.261747 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-8237acf9-8f2d-41e3-8839-acc7f3cf8436 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:59:09.262998 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-8237acf9-8f2d-41e3-8839-acc7f3cf8436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:09.346586 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1254/2501] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:59:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:59:09.368421 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8e2193-d1ff-4121-ab66-248ac1bdb375 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:09.370122 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-ab8e2193-d1ff-4121-ab66-248ac1bdb375 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:09.438210 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:59:09.453919 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8e2193-d1ff-4121-ab66-248ac1bdb375 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:09.458662 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f337e55c-06f8-4107-972d-4e062201e739 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:09.459327 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f337e55c-06f8-4107-972d-4e062201e739 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:09.459644 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f337e55c-06f8-4107-972d-4e062201e739 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:09.460152 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f337e55c-06f8-4107-972d-4e062201e739 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:09.473248 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f337e55c-06f8-4107-972d-4e062201e739 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:09.474731 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f337e55c-06f8-4107-972d-4e062201e739 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:09.487168 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1248/2502] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:59:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:59:10.030704 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8237acf9-8f2d-41e3-8839-acc7f3cf8436 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:10.032587 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-8237acf9-8f2d-41e3-8839-acc7f3cf8436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:10.047830 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8237acf9-8f2d-41e3-8839-acc7f3cf8436 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:10.059318 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4235b4-fa15-4c3f-81b0-8b8ad238352c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:10.059318 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4235b4-fa15-4c3f-81b0-8b8ad238352c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:10.059318 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4235b4-fa15-4c3f-81b0-8b8ad238352c admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:10.059318 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4235b4-fa15-4c3f-81b0-8b8ad238352c admin admin] PATH_INFO: `/v3/projects/dcc34dcadd0d4c2c9d571c81a4714cbc` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:10.059663 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4235b4-fa15-4c3f-81b0-8b8ad238352c admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcc34dcadd0d4c2c9d571c81a4714cbc)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:10.059663 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4235b4-fa15-4c3f-81b0-8b8ad238352c admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:10.204779 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:59:10.235132 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-6e4235b4-fa15-4c3f-81b0-8b8ad238352c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcc34dcadd0d4c2c9d571c81a4714cbc', 'request_id': 'req-6e4235b4-fa15-4c3f-81b0-8b8ad238352c'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:10.245447 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-6e4235b4-fa15-4c3f-81b0-8b8ad238352c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcc34dcadd0d4c2c9d571c81a4714cbc', 'request_id': 'req-6e4235b4-fa15-4c3f-81b0-8b8ad238352c'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:10.334600 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-6e4235b4-fa15-4c3f-81b0-8b8ad238352c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcc34dcadd0d4c2c9d571c81a4714cbc', 'request_id': 'req-6e4235b4-fa15-4c3f-81b0-8b8ad238352c'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:10.349187 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-6e4235b4-fa15-4c3f-81b0-8b8ad238352c admin admin] The token cache is being invalidate because project dcc34dcadd0d4c2c9d571c81a4714cbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:59:10.349510 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-6e4235b4-fa15-4c3f-81b0-8b8ad238352c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:10.350209 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-6e4235b4-fa15-4c3f-81b0-8b8ad238352c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:10.351880 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1255/2503] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:59:10 2022] DELETE /identity/v3/projects/dcc34dcadd0d4c2c9d571c81a4714cbc => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:59:10.355914 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f337e55c-06f8-4107-972d-4e062201e739 service neutron] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:10.367757 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f337e55c-06f8-4107-972d-4e062201e739 service neutron] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:10.439016 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacd63f7-b742-40e0-ae61-eba533fab89a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:10.440270 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacd63f7-b742-40e0-ae61-eba533fab89a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:10.440761 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacd63f7-b742-40e0-ae61-eba533fab89a admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:10.441118 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacd63f7-b742-40e0-ae61-eba533fab89a admin admin] PATH_INFO: `/v3/users/3554e294cb3447f5b6873e3594c5b686` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:10.442442 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacd63f7-b742-40e0-ae61-eba533fab89a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3554e294cb3447f5b6873e3594c5b686)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:10.444601 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacd63f7-b742-40e0-ae61-eba533fab89a admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:10.733848 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-dacd63f7-b742-40e0-ae61-eba533fab89a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3554e294cb3447f5b6873e3594c5b686'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:10.734410 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-dacd63f7-b742-40e0-ae61-eba533fab89a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3554e294cb3447f5b6873e3594c5b686'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:10.742841 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-dacd63f7-b742-40e0-ae61-eba533fab89a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3554e294cb3447f5b6873e3594c5b686'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:10.743563 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-dacd63f7-b742-40e0-ae61-eba533fab89a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3554e294cb3447f5b6873e3594c5b686'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:10.858535 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-dacd63f7-b742-40e0-ae61-eba533fab89a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3554e294cb3447f5b6873e3594c5b686'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:10.876769 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1249/2504] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:59:10 2022] DELETE /identity/v3/users/3554e294cb3447f5b6873e3594c5b686 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:59:11.296124 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4235b4-fa15-4c3f-81b0-8b8ad238352c admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:11.297552 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-6e4235b4-fa15-4c3f-81b0-8b8ad238352c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:11.312069 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4235b4-fa15-4c3f-81b0-8b8ad238352c admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:11.359188 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71a646f-8d3e-4a4b-95d5-31f3b0909389 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:11.360356 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71a646f-8d3e-4a4b-95d5-31f3b0909389 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:11.360744 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71a646f-8d3e-4a4b-95d5-31f3b0909389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:11.387258 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-f71a646f-8d3e-4a4b-95d5-31f3b0909389 None None] MFA Rules not processed for user `19d82c04f9994654adfb10f4bd0573c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:59:11.388721 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-f71a646f-8d3e-4a4b-95d5-31f3b0909389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:11.389173 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e08db2d-308b-4618-a465-435e107d547b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:11.390057 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e08db2d-308b-4618-a465-435e107d547b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:11.390336 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e08db2d-308b-4618-a465-435e107d547b admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:11.390629 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e08db2d-308b-4618-a465-435e107d547b admin admin] PATH_INFO: `/v3/projects/0d2e4a8f03334314817d6343922f999e` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:11.391810 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e08db2d-308b-4618-a465-435e107d547b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d2e4a8f03334314817d6343922f999e)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:11.394326 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e08db2d-308b-4618-a465-435e107d547b admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:11.449062 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1250/2505] 10.222.0.42 () {58 vars in 967 bytes} [Wed Jun 15 14:59:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:59:11.508587 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-2e08db2d-308b-4618-a465-435e107d547b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d2e4a8f03334314817d6343922f999e', 'request_id': 'req-2e08db2d-308b-4618-a465-435e107d547b'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:11.514813 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-2e08db2d-308b-4618-a465-435e107d547b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d2e4a8f03334314817d6343922f999e', 'request_id': 'req-2e08db2d-308b-4618-a465-435e107d547b'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:11.602893 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71a646f-8d3e-4a4b-95d5-31f3b0909389 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:11.604440 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-f71a646f-8d3e-4a4b-95d5-31f3b0909389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:11.613297 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-2e08db2d-308b-4618-a465-435e107d547b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d2e4a8f03334314817d6343922f999e', 'request_id': 'req-2e08db2d-308b-4618-a465-435e107d547b'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:11.621944 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:59:11.624837 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-2e08db2d-308b-4618-a465-435e107d547b admin admin] The token cache is being invalidate because project 0d2e4a8f03334314817d6343922f999e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:59:11.625146 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-2e08db2d-308b-4618-a465-435e107d547b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:11.625777 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-2e08db2d-308b-4618-a465-435e107d547b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:11.627597 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1256/2506] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:59:11 2022] DELETE /identity/v3/projects/0d2e4a8f03334314817d6343922f999e => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:59:11.632484 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264e756a-92d3-43ab-a918-549db330858f None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:11.632771 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264e756a-92d3-43ab-a918-549db330858f None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:11.633051 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264e756a-92d3-43ab-a918-549db330858f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:11.652985 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-264e756a-92d3-43ab-a918-549db330858f None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:59:11.654205 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-264e756a-92d3-43ab-a918-549db330858f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:11.683622 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71a646f-8d3e-4a4b-95d5-31f3b0909389 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:11.698825 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:59:11.718238 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1257/2507] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:59:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:59:11.726180 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb9c5af-11d3-4c33-91c0-b0a3c13b3d0e None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:11.726466 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb9c5af-11d3-4c33-91c0-b0a3c13b3d0e None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:11.726745 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb9c5af-11d3-4c33-91c0-b0a3c13b3d0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:11.744826 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-eeb9c5af-11d3-4c33-91c0-b0a3c13b3d0e None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:59:11.746035 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-eeb9c5af-11d3-4c33-91c0-b0a3c13b3d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:11.747582 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dddd2ea-385d-4540-ade8-29f50605aff4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04dc5832333498e96a7e02bbfb9484c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:11.748369 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dddd2ea-385d-4540-ade8-29f50605aff4 service swift] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:11.753879 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dddd2ea-385d-4540-ade8-29f50605aff4 service swift] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:11.753879 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dddd2ea-385d-4540-ade8-29f50605aff4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:11.753879 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-2dddd2ea-385d-4540-ade8-29f50605aff4 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:11.755221 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1258/2508] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:59:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:59:11.759659 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb9c5af-11d3-4c33-91c0-b0a3c13b3d0e None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:11.769842 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb9c5af-11d3-4c33-91c0-b0a3c13b3d0e None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:11.775613 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f620b1c8-7e49-4c13-b9dc-aec84c08fdb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:11.776707 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f620b1c8-7e49-4c13-b9dc-aec84c08fdb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:11.777103 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f620b1c8-7e49-4c13-b9dc-aec84c08fdb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:11.777345 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f620b1c8-7e49-4c13-b9dc-aec84c08fdb1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:11.778094 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f620b1c8-7e49-4c13-b9dc-aec84c08fdb1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:59:11.778406 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f620b1c8-7e49-4c13-b9dc-aec84c08fdb1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:11.780062 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f620b1c8-7e49-4c13-b9dc-aec84c08fdb1 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:11.796539 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1259/2509] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jun 15 14:59:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:59:11.799034 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f620b1c8-7e49-4c13-b9dc-aec84c08fdb1 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:11.814801 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f620b1c8-7e49-4c13-b9dc-aec84c08fdb1 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:11.825541 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70fee17-6251-4bdd-84a9-81c3dd027818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:11.826488 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dddd2ea-385d-4540-ade8-29f50605aff4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:11.827427 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70fee17-6251-4bdd-84a9-81c3dd027818 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:11.827671 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dddd2ea-385d-4540-ade8-29f50605aff4 service swift] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:11.828376 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70fee17-6251-4bdd-84a9-81c3dd027818 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:11.829550 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70fee17-6251-4bdd-84a9-81c3dd027818 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:11.832946 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70fee17-6251-4bdd-84a9-81c3dd027818 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:11.835970 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70fee17-6251-4bdd-84a9-81c3dd027818 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:11.848330 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1251/2510] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Jun 15 14:59:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 15 14:59:11.955377 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1260/2511] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:59:11 2022] POST /identity/v3/projects => generated 385 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:59:11.960241 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dddd2ea-385d-4540-ade8-29f50605aff4 service swift] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:11.970863 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dddd2ea-385d-4540-ade8-29f50605aff4 service swift] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:12.024285 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:59:12.034199 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878d08b4-b2ed-4834-9763-cf62dbb1906b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:12.035717 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d08b4-b2ed-4834-9763-cf62dbb1906b admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:12.036054 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d08b4-b2ed-4834-9763-cf62dbb1906b admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:12.036363 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d08b4-b2ed-4834-9763-cf62dbb1906b admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:12.037101 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878d08b4-b2ed-4834-9763-cf62dbb1906b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:12.039047 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878d08b4-b2ed-4834-9763-cf62dbb1906b admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:12.213530 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1252/2512] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:59:11 2022] POST /identity/v3/users => generated 293 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:59:12.214773 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70fee17-6251-4bdd-84a9-81c3dd027818 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:12.226242 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70fee17-6251-4bdd-84a9-81c3dd027818 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:12.231125 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e810761-4033-4b9d-9e1d-3f5383ebe3eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:12.235574 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e810761-4033-4b9d-9e1d-3f5383ebe3eb admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:12.235574 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e810761-4033-4b9d-9e1d-3f5383ebe3eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:12.235574 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e810761-4033-4b9d-9e1d-3f5383ebe3eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:12.235574 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e810761-4033-4b9d-9e1d-3f5383ebe3eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:12.235574 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e810761-4033-4b9d-9e1d-3f5383ebe3eb admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:12.245764 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1261/2513] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:59:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:59:12.250007 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878d08b4-b2ed-4834-9763-cf62dbb1906b admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:12.260549 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878d08b4-b2ed-4834-9763-cf62dbb1906b admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:12.265343 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43061fb5-985f-4ac4-8cf4-a8f278616fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:12.265998 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43061fb5-985f-4ac4-8cf4-a8f278616fc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:12.266285 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43061fb5-985f-4ac4-8cf4-a8f278616fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:12.266561 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43061fb5-985f-4ac4-8cf4-a8f278616fc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:12.267349 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43061fb5-985f-4ac4-8cf4-a8f278616fc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:12.269508 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43061fb5-985f-4ac4-8cf4-a8f278616fc4 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:12.279873 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1253/2514] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:59:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:59:12.284130 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e810761-4033-4b9d-9e1d-3f5383ebe3eb admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:12.296634 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e810761-4033-4b9d-9e1d-3f5383ebe3eb admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:12.301501 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a889d2b3-b3f8-4ec8-bd78-4c3af8fabb59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:12.302322 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a889d2b3-b3f8-4ec8-bd78-4c3af8fabb59 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:12.302607 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a889d2b3-b3f8-4ec8-bd78-4c3af8fabb59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:12.303140 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a889d2b3-b3f8-4ec8-bd78-4c3af8fabb59 admin admin] PATH_INFO: `/v3/projects/4923414450984b6e84fc133df66d9c74/users/428979b451f040d9b273fbea0ffbe893/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:12.318693 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a889d2b3-b3f8-4ec8-bd78-4c3af8fabb59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4923414450984b6e84fc133df66d9c74, user_id=428979b451f040d9b273fbea0ffbe893, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:12.322393 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a889d2b3-b3f8-4ec8-bd78-4c3af8fabb59 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:12.500499 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1262/2515] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:59:12 2022] PUT /identity/v3/projects/4923414450984b6e84fc133df66d9c74/users/428979b451f040d9b273fbea0ffbe893/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:59:12.511563 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ff5fb-23a4-4cc0-b817-44ce1de20834 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:12.512050 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ff5fb-23a4-4cc0-b817-44ce1de20834 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:12.512188 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ff5fb-23a4-4cc0-b817-44ce1de20834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:12.540782 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-de3ff5fb-23a4-4cc0-b817-44ce1de20834 None None] MFA Rules not processed for user `428979b451f040d9b273fbea0ffbe893`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:59:12.545914 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-de3ff5fb-23a4-4cc0-b817-44ce1de20834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:12.616776 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1254/2516] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:59:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:59:12.621451 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a889d2b3-b3f8-4ec8-bd78-4c3af8fabb59 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:12.687501 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a889d2b3-b3f8-4ec8-bd78-4c3af8fabb59 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:12.693044 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69caad2a-d7a1-48ca-af0f-63e7349bfcb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:12.693929 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69caad2a-d7a1-48ca-af0f-63e7349bfcb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:12.694210 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69caad2a-d7a1-48ca-af0f-63e7349bfcb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:12.694483 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69caad2a-d7a1-48ca-af0f-63e7349bfcb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:12.695170 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69caad2a-d7a1-48ca-af0f-63e7349bfcb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:12.697145 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69caad2a-d7a1-48ca-af0f-63e7349bfcb2 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:12.839280 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1263/2517] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:59:12 2022] POST /identity/v3/projects => generated 383 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:59:12.843577 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3ff5fb-23a4-4cc0-b817-44ce1de20834 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:12.854789 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3ff5fb-23a4-4cc0-b817-44ce1de20834 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:12.915419 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d0de3e-a72b-4cda-9a41-d720481cae62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:12.916125 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d0de3e-a72b-4cda-9a41-d720481cae62 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:12.916413 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d0de3e-a72b-4cda-9a41-d720481cae62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:12.916693 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d0de3e-a72b-4cda-9a41-d720481cae62 admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:12.917399 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d0de3e-a72b-4cda-9a41-d720481cae62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:12.919320 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d0de3e-a72b-4cda-9a41-d720481cae62 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:13.050394 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1255/2518] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:59:12 2022] POST /identity/v3/users => generated 298 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:59:13.054450 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69caad2a-d7a1-48ca-af0f-63e7349bfcb2 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:13.065010 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69caad2a-d7a1-48ca-af0f-63e7349bfcb2 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:13.070400 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702ab528-f47c-4a32-bb38-76a30175e2c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:13.070937 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ab528-f47c-4a32-bb38-76a30175e2c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:13.071222 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ab528-f47c-4a32-bb38-76a30175e2c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:13.071498 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ab528-f47c-4a32-bb38-76a30175e2c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:13.072257 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702ab528-f47c-4a32-bb38-76a30175e2c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:13.073848 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702ab528-f47c-4a32-bb38-76a30175e2c1 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:13.085953 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1264/2519] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:59:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:59:13.090138 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d0de3e-a72b-4cda-9a41-d720481cae62 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:13.101016 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d0de3e-a72b-4cda-9a41-d720481cae62 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:13.106077 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e687b848-6755-4067-ad63-f7340fd3ec01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:13.106984 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e687b848-6755-4067-ad63-f7340fd3ec01 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:13.107274 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e687b848-6755-4067-ad63-f7340fd3ec01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:13.107552 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e687b848-6755-4067-ad63-f7340fd3ec01 admin admin] PATH_INFO: `/v3/projects/c5ce4a03baed4c4bb384df86272b5c35/users/f7818174051a43c8a5b2ec2c21f279e7/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:13.123559 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e687b848-6755-4067-ad63-f7340fd3ec01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5ce4a03baed4c4bb384df86272b5c35, user_id=f7818174051a43c8a5b2ec2c21f279e7, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:13.126356 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e687b848-6755-4067-ad63-f7340fd3ec01 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:13.218908 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1256/2520] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:59:13 2022] PUT /identity/v3/projects/c5ce4a03baed4c4bb384df86272b5c35/users/f7818174051a43c8a5b2ec2c21f279e7/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:59:13.223916 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702ab528-f47c-4a32-bb38-76a30175e2c1 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:13.234283 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702ab528-f47c-4a32-bb38-76a30175e2c1 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:13.276057 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:59:13.292374 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea8fe00-b7c5-4ca7-8698-ac0938f0d46a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:13.293082 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea8fe00-b7c5-4ca7-8698-ac0938f0d46a admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:13.293508 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea8fe00-b7c5-4ca7-8698-ac0938f0d46a admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:13.293814 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea8fe00-b7c5-4ca7-8698-ac0938f0d46a admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:13.294326 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea8fe00-b7c5-4ca7-8698-ac0938f0d46a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:13.295954 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea8fe00-b7c5-4ca7-8698-ac0938f0d46a admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:13.307073 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1265/2521] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:59:13 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:59:13.312627 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e687b848-6755-4067-ad63-f7340fd3ec01 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:13.328201 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e687b848-6755-4067-ad63-f7340fd3ec01 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:13.333936 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871a8f38-dfc4-4703-ba7b-c8204ba679c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:13.336735 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871a8f38-dfc4-4703-ba7b-c8204ba679c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:13.336735 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871a8f38-dfc4-4703-ba7b-c8204ba679c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:13.336735 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871a8f38-dfc4-4703-ba7b-c8204ba679c8 admin admin] PATH_INFO: `/v3/projects/c5ce4a03baed4c4bb384df86272b5c35/users/f7818174051a43c8a5b2ec2c21f279e7/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:13.339897 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871a8f38-dfc4-4703-ba7b-c8204ba679c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5ce4a03baed4c4bb384df86272b5c35, user_id=f7818174051a43c8a5b2ec2c21f279e7, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:13.345377 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871a8f38-dfc4-4703-ba7b-c8204ba679c8 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:13.365751 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1257/2522] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:59:13 2022] PUT /identity/v3/projects/c5ce4a03baed4c4bb384df86272b5c35/users/f7818174051a43c8a5b2ec2c21f279e7/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:59:13.385431 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1842e2a0-be69-4742-ab9a-eba6ef3281dd None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:13.386046 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1842e2a0-be69-4742-ab9a-eba6ef3281dd None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:13.386507 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1842e2a0-be69-4742-ab9a-eba6ef3281dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:13.414977 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-1842e2a0-be69-4742-ab9a-eba6ef3281dd None None] MFA Rules not processed for user `f7818174051a43c8a5b2ec2c21f279e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:59:13.416418 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-1842e2a0-be69-4742-ab9a-eba6ef3281dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:13.494970 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1266/2523] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:59:13 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:59:13.507488 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871a8f38-dfc4-4703-ba7b-c8204ba679c8 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:13.512246 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-871a8f38-dfc4-4703-ba7b-c8204ba679c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:13.524580 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:59:13.574257 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871a8f38-dfc4-4703-ba7b-c8204ba679c8 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:13.577971 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64700ff2-8234-46e3-aafd-f544188f25fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:13.578708 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64700ff2-8234-46e3-aafd-f544188f25fe service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:13.578995 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64700ff2-8234-46e3-aafd-f544188f25fe service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:13.579301 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64700ff2-8234-46e3-aafd-f544188f25fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:13.653623 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64700ff2-8234-46e3-aafd-f544188f25fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:13.655115 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64700ff2-8234-46e3-aafd-f544188f25fe service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:13.666345 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1258/2524] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:59:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:59:17.553577 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1842e2a0-be69-4742-ab9a-eba6ef3281dd None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:17.555686 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-1842e2a0-be69-4742-ab9a-eba6ef3281dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:17.634922 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1842e2a0-be69-4742-ab9a-eba6ef3281dd None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:17.639998 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478dd583-8b10-43d3-bc8a-10623ff04e29 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bb6671c19464424a6e9cd4519ac93e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:17.640673 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478dd583-8b10-43d3-bc8a-10623ff04e29 service placement] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:17.640951 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478dd583-8b10-43d3-bc8a-10623ff04e29 service placement] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:17.641267 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478dd583-8b10-43d3-bc8a-10623ff04e29 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:17.643167 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-478dd583-8b10-43d3-bc8a-10623ff04e29 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:17.655472 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478dd583-8b10-43d3-bc8a-10623ff04e29 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:17.657021 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478dd583-8b10-43d3-bc8a-10623ff04e29 service placement] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:17.667887 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1267/2525] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:59:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:59:20.419029 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64700ff2-8234-46e3-aafd-f544188f25fe service cinder] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:20.420613 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-64700ff2-8234-46e3-aafd-f544188f25fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:20.483774 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64700ff2-8234-46e3-aafd-f544188f25fe service cinder] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:20.487988 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533adf81-bff7-4152-aa89-009f7482d66d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:20.488671 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533adf81-bff7-4152-aa89-009f7482d66d service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:20.488933 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533adf81-bff7-4152-aa89-009f7482d66d service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:20.489211 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533adf81-bff7-4152-aa89-009f7482d66d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:20.491085 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-533adf81-bff7-4152-aa89-009f7482d66d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:20.571860 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533adf81-bff7-4152-aa89-009f7482d66d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:20.573355 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533adf81-bff7-4152-aa89-009f7482d66d service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:20.583596 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1259/2526] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:59:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:59:21.671053 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478dd583-8b10-43d3-bc8a-10623ff04e29 service placement] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:21.672714 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-478dd583-8b10-43d3-bc8a-10623ff04e29 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:21.688563 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478dd583-8b10-43d3-bc8a-10623ff04e29 service placement] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:21.694345 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79067ea-959a-41a9-8f46-65823d5e45aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:21.695208 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79067ea-959a-41a9-8f46-65823d5e45aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:21.695494 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79067ea-959a-41a9-8f46-65823d5e45aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:21.695774 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79067ea-959a-41a9-8f46-65823d5e45aa admin admin] PATH_INFO: `/v3/users/8a8c023cd5014daa993be2735b1c05ab` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:21.697006 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79067ea-959a-41a9-8f46-65823d5e45aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a8c023cd5014daa993be2735b1c05ab)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:21.698948 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79067ea-959a-41a9-8f46-65823d5e45aa admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:21.896908 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-e79067ea-959a-41a9-8f46-65823d5e45aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a8c023cd5014daa993be2735b1c05ab'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:21.897635 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-e79067ea-959a-41a9-8f46-65823d5e45aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a8c023cd5014daa993be2735b1c05ab'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:21.898303 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-e79067ea-959a-41a9-8f46-65823d5e45aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a8c023cd5014daa993be2735b1c05ab'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:21.900271 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:59:21.996379 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-e79067ea-959a-41a9-8f46-65823d5e45aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a8c023cd5014daa993be2735b1c05ab'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:22.009532 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-e79067ea-959a-41a9-8f46-65823d5e45aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a8c023cd5014daa993be2735b1c05ab'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:22.022529 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1268/2527] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:59:21 2022] DELETE /identity/v3/users/8a8c023cd5014daa993be2735b1c05ab => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:59:22.187177 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533adf81-bff7-4152-aa89-009f7482d66d service neutron] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:22.197452 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533adf81-bff7-4152-aa89-009f7482d66d service neutron] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:22.210113 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:59:22.247110 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c773bd3-703d-42c7-bb5d-95234d78a8f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:22.247760 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c773bd3-703d-42c7-bb5d-95234d78a8f8 service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:22.248043 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c773bd3-703d-42c7-bb5d-95234d78a8f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:22.248353 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c773bd3-703d-42c7-bb5d-95234d78a8f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:22.250113 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-1c773bd3-703d-42c7-bb5d-95234d78a8f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:22.314514 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79067ea-959a-41a9-8f46-65823d5e45aa admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:22.317876 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e79067ea-959a-41a9-8f46-65823d5e45aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:22.326789 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c773bd3-703d-42c7-bb5d-95234d78a8f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:22.328274 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c773bd3-703d-42c7-bb5d-95234d78a8f8 service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:22.340493 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1260/2528] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:59:22 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:59:22.418006 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79067ea-959a-41a9-8f46-65823d5e45aa admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:22.418006 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8ef6bf-8c1c-4ca2-9a97-5eeb066c6156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:22.418006 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8ef6bf-8c1c-4ca2-9a97-5eeb066c6156 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:22.418006 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8ef6bf-8c1c-4ca2-9a97-5eeb066c6156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:22.418401 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8ef6bf-8c1c-4ca2-9a97-5eeb066c6156 admin admin] PATH_INFO: `/v3/projects/7a96a40d9feb40b8bbc143f1f9be7408` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:22.418897 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8ef6bf-8c1c-4ca2-9a97-5eeb066c6156 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a96a40d9feb40b8bbc143f1f9be7408)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:22.420873 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8ef6bf-8c1c-4ca2-9a97-5eeb066c6156 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:22.559046 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-cc8ef6bf-8c1c-4ca2-9a97-5eeb066c6156 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a96a40d9feb40b8bbc143f1f9be7408', 'request_id': 'req-cc8ef6bf-8c1c-4ca2-9a97-5eeb066c6156'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:22.566089 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-cc8ef6bf-8c1c-4ca2-9a97-5eeb066c6156 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a96a40d9feb40b8bbc143f1f9be7408', 'request_id': 'req-cc8ef6bf-8c1c-4ca2-9a97-5eeb066c6156'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:22.630756 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-cc8ef6bf-8c1c-4ca2-9a97-5eeb066c6156 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a96a40d9feb40b8bbc143f1f9be7408', 'request_id': 'req-cc8ef6bf-8c1c-4ca2-9a97-5eeb066c6156'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:22.642220 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-cc8ef6bf-8c1c-4ca2-9a97-5eeb066c6156 admin admin] The token cache is being invalidate because project 7a96a40d9feb40b8bbc143f1f9be7408 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:59:22.644939 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-cc8ef6bf-8c1c-4ca2-9a97-5eeb066c6156 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:22.644939 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-cc8ef6bf-8c1c-4ca2-9a97-5eeb066c6156 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:22.645462 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1269/2529] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:59:22 2022] DELETE /identity/v3/projects/7a96a40d9feb40b8bbc143f1f9be7408 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:59:22.649570 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c773bd3-703d-42c7-bb5d-95234d78a8f8 service cinder] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:22.660007 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c773bd3-703d-42c7-bb5d-95234d78a8f8 service cinder] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:22.728012 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba75b6ef-34e7-4e0c-95d5-8f0c3a408512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:22.728662 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba75b6ef-34e7-4e0c-95d5-8f0c3a408512 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:22.728938 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba75b6ef-34e7-4e0c-95d5-8f0c3a408512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:22.729208 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba75b6ef-34e7-4e0c-95d5-8f0c3a408512 admin admin] PATH_INFO: `/v3/users/668d27518c794b3cadbb76dc628e0683` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:22.730444 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba75b6ef-34e7-4e0c-95d5-8f0c3a408512 admin admin] RBAC: Authorizing `identity:delete_user(user_id=668d27518c794b3cadbb76dc628e0683)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:22.732383 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba75b6ef-34e7-4e0c-95d5-8f0c3a408512 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:22.972873 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-ba75b6ef-34e7-4e0c-95d5-8f0c3a408512 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '668d27518c794b3cadbb76dc628e0683'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:22.975053 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-ba75b6ef-34e7-4e0c-95d5-8f0c3a408512 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '668d27518c794b3cadbb76dc628e0683'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:22.982391 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-ba75b6ef-34e7-4e0c-95d5-8f0c3a408512 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '668d27518c794b3cadbb76dc628e0683'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:22.983040 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-ba75b6ef-34e7-4e0c-95d5-8f0c3a408512 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '668d27518c794b3cadbb76dc628e0683'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:23.060152 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-ba75b6ef-34e7-4e0c-95d5-8f0c3a408512 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '668d27518c794b3cadbb76dc628e0683'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:23.067112 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:59:23.082070 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1261/2530] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:59:22 2022] DELETE /identity/v3/users/668d27518c794b3cadbb76dc628e0683 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:59:23.376345 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8ef6bf-8c1c-4ca2-9a97-5eeb066c6156 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:23.378244 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-cc8ef6bf-8c1c-4ca2-9a97-5eeb066c6156 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:23.392823 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8ef6bf-8c1c-4ca2-9a97-5eeb066c6156 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:23.453799 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:59:23.461197 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb230ae-f182-4ff6-a21e-487076eb28f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:23.462048 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb230ae-f182-4ff6-a21e-487076eb28f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:23.462330 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb230ae-f182-4ff6-a21e-487076eb28f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:23.462603 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb230ae-f182-4ff6-a21e-487076eb28f4 admin admin] PATH_INFO: `/v3/projects/0560d2bbfd424594bd60ff80f99dcb14` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:23.463916 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb230ae-f182-4ff6-a21e-487076eb28f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0560d2bbfd424594bd60ff80f99dcb14)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:23.466155 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb230ae-f182-4ff6-a21e-487076eb28f4 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:23.575891 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-4eb230ae-f182-4ff6-a21e-487076eb28f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0560d2bbfd424594bd60ff80f99dcb14', 'request_id': 'req-4eb230ae-f182-4ff6-a21e-487076eb28f4'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:23.583170 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-4eb230ae-f182-4ff6-a21e-487076eb28f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0560d2bbfd424594bd60ff80f99dcb14', 'request_id': 'req-4eb230ae-f182-4ff6-a21e-487076eb28f4'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:23.659912 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-4eb230ae-f182-4ff6-a21e-487076eb28f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0560d2bbfd424594bd60ff80f99dcb14', 'request_id': 'req-4eb230ae-f182-4ff6-a21e-487076eb28f4'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:23.671632 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-4eb230ae-f182-4ff6-a21e-487076eb28f4 admin admin] The token cache is being invalidate because project 0560d2bbfd424594bd60ff80f99dcb14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:59:23.672039 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-4eb230ae-f182-4ff6-a21e-487076eb28f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:23.673132 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-4eb230ae-f182-4ff6-a21e-487076eb28f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:23.674762 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1270/2531] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:59:23 2022] DELETE /identity/v3/projects/0560d2bbfd424594bd60ff80f99dcb14 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:59:29.433066 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba75b6ef-34e7-4e0c-95d5-8f0c3a408512 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:29.434543 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-ba75b6ef-34e7-4e0c-95d5-8f0c3a408512 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:29.448072 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba75b6ef-34e7-4e0c-95d5-8f0c3a408512 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:29.510672 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4f4b93-f72f-4168-8e94-129d43ed66e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:29.511674 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4f4b93-f72f-4168-8e94-129d43ed66e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:29.512106 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4f4b93-f72f-4168-8e94-129d43ed66e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:29.519783 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4f4b93-f72f-4168-8e94-129d43ed66e7 admin admin] PATH_INFO: `/v3/users/428979b451f040d9b273fbea0ffbe893` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:29.519783 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4f4b93-f72f-4168-8e94-129d43ed66e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=428979b451f040d9b273fbea0ffbe893)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:29.519783 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4f4b93-f72f-4168-8e94-129d43ed66e7 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:29.714309 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-9c4f4b93-f72f-4168-8e94-129d43ed66e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '428979b451f040d9b273fbea0ffbe893'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:29.714897 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-9c4f4b93-f72f-4168-8e94-129d43ed66e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '428979b451f040d9b273fbea0ffbe893'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:29.721849 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-9c4f4b93-f72f-4168-8e94-129d43ed66e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '428979b451f040d9b273fbea0ffbe893'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:29.722467 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-9c4f4b93-f72f-4168-8e94-129d43ed66e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '428979b451f040d9b273fbea0ffbe893'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:29.808497 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-9c4f4b93-f72f-4168-8e94-129d43ed66e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '428979b451f040d9b273fbea0ffbe893'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:29.827426 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1262/2532] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:59:29 2022] DELETE /identity/v3/users/428979b451f040d9b273fbea0ffbe893 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:59:29.831992 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29358292-b897-49fa-8331-90e05b99bab9 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:29.832631 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29358292-b897-49fa-8331-90e05b99bab9 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:29.832631 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29358292-b897-49fa-8331-90e05b99bab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:29.850834 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-29358292-b897-49fa-8331-90e05b99bab9 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:59:29.852109 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-29358292-b897-49fa-8331-90e05b99bab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:29.912335 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1271/2533] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:59:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:59:29.922272 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4f4b93-f72f-4168-8e94-129d43ed66e7 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:29.923872 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-9c4f4b93-f72f-4168-8e94-129d43ed66e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:29.984846 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4f4b93-f72f-4168-8e94-129d43ed66e7 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:29.988721 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d849d4-6b0c-4722-91b8-f13d55e2a99b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:29.989507 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d849d4-6b0c-4722-91b8-f13d55e2a99b service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:29.989841 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d849d4-6b0c-4722-91b8-f13d55e2a99b service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:29.990129 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d849d4-6b0c-4722-91b8-f13d55e2a99b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:29.999553 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d849d4-6b0c-4722-91b8-f13d55e2a99b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:30.001017 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d849d4-6b0c-4722-91b8-f13d55e2a99b service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:30.011135 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1263/2534] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:59:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:59:30.501985 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29358292-b897-49fa-8331-90e05b99bab9 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:30.503401 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-29358292-b897-49fa-8331-90e05b99bab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:30.517204 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29358292-b897-49fa-8331-90e05b99bab9 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:30.523076 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9160210e-f248-4065-805c-039f88e84961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:30.523946 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9160210e-f248-4065-805c-039f88e84961 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:30.524267 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9160210e-f248-4065-805c-039f88e84961 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:30.524547 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9160210e-f248-4065-805c-039f88e84961 admin admin] PATH_INFO: `/v3/projects/4923414450984b6e84fc133df66d9c74` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:30.525824 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9160210e-f248-4065-805c-039f88e84961 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4923414450984b6e84fc133df66d9c74)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:30.527896 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9160210e-f248-4065-805c-039f88e84961 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:30.667680 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-9160210e-f248-4065-805c-039f88e84961 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4923414450984b6e84fc133df66d9c74', 'request_id': 'req-9160210e-f248-4065-805c-039f88e84961'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:30.674139 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-9160210e-f248-4065-805c-039f88e84961 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4923414450984b6e84fc133df66d9c74', 'request_id': 'req-9160210e-f248-4065-805c-039f88e84961'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:30.751799 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-9160210e-f248-4065-805c-039f88e84961 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4923414450984b6e84fc133df66d9c74', 'request_id': 'req-9160210e-f248-4065-805c-039f88e84961'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:30.764133 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-9160210e-f248-4065-805c-039f88e84961 admin admin] The token cache is being invalidate because project 4923414450984b6e84fc133df66d9c74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:59:30.764426 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-9160210e-f248-4065-805c-039f88e84961 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:30.765085 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-9160210e-f248-4065-805c-039f88e84961 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:30.766696 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1272/2535] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:59:30 2022] DELETE /identity/v3/projects/4923414450984b6e84fc133df66d9c74 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:59:30.770852 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d849d4-6b0c-4722-91b8-f13d55e2a99b service neutron] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:30.780843 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d849d4-6b0c-4722-91b8-f13d55e2a99b service neutron] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:30.784561 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:59:30.838621 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b9140-5131-44ff-8818-d4c3b882a31b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:30.839611 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b9140-5131-44ff-8818-d4c3b882a31b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:30.839895 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b9140-5131-44ff-8818-d4c3b882a31b admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:30.840204 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b9140-5131-44ff-8818-d4c3b882a31b admin admin] PATH_INFO: `/v3/users/f7818174051a43c8a5b2ec2c21f279e7` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:30.841407 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624b9140-5131-44ff-8818-d4c3b882a31b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7818174051a43c8a5b2ec2c21f279e7)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:30.843317 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624b9140-5131-44ff-8818-d4c3b882a31b admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:31.042052 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-624b9140-5131-44ff-8818-d4c3b882a31b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7818174051a43c8a5b2ec2c21f279e7'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:31.042632 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-624b9140-5131-44ff-8818-d4c3b882a31b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7818174051a43c8a5b2ec2c21f279e7'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:31.049725 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-624b9140-5131-44ff-8818-d4c3b882a31b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7818174051a43c8a5b2ec2c21f279e7'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:31.050366 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-624b9140-5131-44ff-8818-d4c3b882a31b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7818174051a43c8a5b2ec2c21f279e7'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:31.116612 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-624b9140-5131-44ff-8818-d4c3b882a31b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7818174051a43c8a5b2ec2c21f279e7'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:31.135551 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1264/2536] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:59:30 2022] DELETE /identity/v3/users/f7818174051a43c8a5b2ec2c21f279e7 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:59:31.581576 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9160210e-f248-4065-805c-039f88e84961 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:31.583065 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-9160210e-f248-4065-805c-039f88e84961 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:31.596263 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9160210e-f248-4065-805c-039f88e84961 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:31.601661 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:59:31.657071 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36f08d4-0f74-43f3-862a-c6060ddbb0d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:31.658100 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36f08d4-0f74-43f3-862a-c6060ddbb0d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:31.658385 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36f08d4-0f74-43f3-862a-c6060ddbb0d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:31.658681 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36f08d4-0f74-43f3-862a-c6060ddbb0d1 admin admin] PATH_INFO: `/v3/projects/c5ce4a03baed4c4bb384df86272b5c35` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:31.659861 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36f08d4-0f74-43f3-862a-c6060ddbb0d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5ce4a03baed4c4bb384df86272b5c35)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:31.662255 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36f08d4-0f74-43f3-862a-c6060ddbb0d1 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:31.821327 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-f36f08d4-0f74-43f3-862a-c6060ddbb0d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5ce4a03baed4c4bb384df86272b5c35', 'request_id': 'req-f36f08d4-0f74-43f3-862a-c6060ddbb0d1'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:31.827711 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-f36f08d4-0f74-43f3-862a-c6060ddbb0d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5ce4a03baed4c4bb384df86272b5c35', 'request_id': 'req-f36f08d4-0f74-43f3-862a-c6060ddbb0d1'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:31.911711 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-f36f08d4-0f74-43f3-862a-c6060ddbb0d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5ce4a03baed4c4bb384df86272b5c35', 'request_id': 'req-f36f08d4-0f74-43f3-862a-c6060ddbb0d1'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:31.923344 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-f36f08d4-0f74-43f3-862a-c6060ddbb0d1 admin admin] The token cache is being invalidate because project c5ce4a03baed4c4bb384df86272b5c35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:59:31.923636 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-f36f08d4-0f74-43f3-862a-c6060ddbb0d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:31.924308 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-f36f08d4-0f74-43f3-862a-c6060ddbb0d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:31.925997 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1273/2537] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:59:31 2022] DELETE /identity/v3/projects/c5ce4a03baed4c4bb384df86272b5c35 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:59:31.933033 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8554f8-4f23-47e4-8ede-174153417480 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:31.933600 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8554f8-4f23-47e4-8ede-174153417480 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:31.933600 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8554f8-4f23-47e4-8ede-174153417480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:31.952266 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-5d8554f8-4f23-47e4-8ede-174153417480 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:59:31.953464 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-5d8554f8-4f23-47e4-8ede-174153417480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:32.015803 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1265/2538] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:59:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:59:32.024462 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf8d442-ea03-481e-8a6a-638066a7b83a None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:32.025013 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf8d442-ea03-481e-8a6a-638066a7b83a None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:32.025488 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf8d442-ea03-481e-8a6a-638066a7b83a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:32.046254 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-eaf8d442-ea03-481e-8a6a-638066a7b83a None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:59:32.047468 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-eaf8d442-ea03-481e-8a6a-638066a7b83a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:32.056523 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1274/2539] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:59:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:59:32.060923 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8554f8-4f23-47e4-8ede-174153417480 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:32.070776 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8554f8-4f23-47e4-8ede-174153417480 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:32.079941 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94503608-dcf7-4db9-8348-26d735823d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:32.079941 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94503608-dcf7-4db9-8348-26d735823d67 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:32.079941 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94503608-dcf7-4db9-8348-26d735823d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:32.079941 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94503608-dcf7-4db9-8348-26d735823d67 admin admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:32.080374 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94503608-dcf7-4db9-8348-26d735823d67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:59:32.080374 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94503608-dcf7-4db9-8348-26d735823d67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:32.080374 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94503608-dcf7-4db9-8348-26d735823d67 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:32.091493 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1266/2540] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jun 15 14:59:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:59:32.095944 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf8d442-ea03-481e-8a6a-638066a7b83a None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:32.106967 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf8d442-ea03-481e-8a6a-638066a7b83a None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:32.111789 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1594d545-334e-47fc-b3e2-a1cb3f30577a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:32.112483 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1594d545-334e-47fc-b3e2-a1cb3f30577a admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:32.112795 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1594d545-334e-47fc-b3e2-a1cb3f30577a admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:32.113065 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1594d545-334e-47fc-b3e2-a1cb3f30577a admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:32.114295 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1594d545-334e-47fc-b3e2-a1cb3f30577a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:32.116559 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1594d545-334e-47fc-b3e2-a1cb3f30577a admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:32.220662 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1275/2541] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:59:32 2022] POST /identity/v3/projects => generated 377 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:59:32.225395 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94503608-dcf7-4db9-8348-26d735823d67 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:32.235639 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94503608-dcf7-4db9-8348-26d735823d67 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:32.244996 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:59:32.294520 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbfd964-11a0-4850-9a76-57637f7baa0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:32.295194 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbfd964-11a0-4850-9a76-57637f7baa0f admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:32.295483 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbfd964-11a0-4850-9a76-57637f7baa0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:32.295763 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbfd964-11a0-4850-9a76-57637f7baa0f admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:32.296490 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbfd964-11a0-4850-9a76-57637f7baa0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:32.298472 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbfd964-11a0-4850-9a76-57637f7baa0f admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:32.403993 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1267/2542] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:59:32 2022] POST /identity/v3/users => generated 289 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:59:32.408051 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1594d545-334e-47fc-b3e2-a1cb3f30577a admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:32.420161 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1594d545-334e-47fc-b3e2-a1cb3f30577a admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:32.424834 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7e8c85-0914-4a51-bc21-f7e892feefc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:32.425485 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7e8c85-0914-4a51-bc21-f7e892feefc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:32.425766 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7e8c85-0914-4a51-bc21-f7e892feefc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:32.426037 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7e8c85-0914-4a51-bc21-f7e892feefc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:32.426778 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7e8c85-0914-4a51-bc21-f7e892feefc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:32.428553 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7e8c85-0914-4a51-bc21-f7e892feefc5 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:32.439644 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1276/2543] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:59:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:59:32.443922 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbfd964-11a0-4850-9a76-57637f7baa0f admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:32.454160 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbfd964-11a0-4850-9a76-57637f7baa0f admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:32.459020 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7830bf2-6a06-411e-aaf2-1267e9ed099e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:32.460395 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7830bf2-6a06-411e-aaf2-1267e9ed099e admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:32.460677 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7830bf2-6a06-411e-aaf2-1267e9ed099e admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:32.460954 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7830bf2-6a06-411e-aaf2-1267e9ed099e admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:32.461681 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7830bf2-6a06-411e-aaf2-1267e9ed099e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:32.463272 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7830bf2-6a06-411e-aaf2-1267e9ed099e admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:32.473401 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1268/2544] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:59:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:59:32.478373 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7e8c85-0914-4a51-bc21-f7e892feefc5 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:32.491180 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7e8c85-0914-4a51-bc21-f7e892feefc5 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:32.495873 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ece72-52a1-4a6f-ba2a-b4bf0a95da07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:32.496798 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ece72-52a1-4a6f-ba2a-b4bf0a95da07 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:32.497086 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ece72-52a1-4a6f-ba2a-b4bf0a95da07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:32.497370 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ece72-52a1-4a6f-ba2a-b4bf0a95da07 admin admin] PATH_INFO: `/v3/projects/15e2a5209f0541008624c0e91e4c5866/users/15e3909f227543819e8c0ba513b5a41a/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:32.513931 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9ece72-52a1-4a6f-ba2a-b4bf0a95da07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15e2a5209f0541008624c0e91e4c5866, user_id=15e3909f227543819e8c0ba513b5a41a, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:32.516651 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9ece72-52a1-4a6f-ba2a-b4bf0a95da07 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:32.625759 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1277/2545] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:59:32 2022] PUT /identity/v3/projects/15e2a5209f0541008624c0e91e4c5866/users/15e3909f227543819e8c0ba513b5a41a/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:59:32.638324 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb7910-b541-4161-9bd6-b7785c47de1f None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:32.638618 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb7910-b541-4161-9bd6-b7785c47de1f None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:32.638906 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb7910-b541-4161-9bd6-b7785c47de1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:32.667570 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-48bb7910-b541-4161-9bd6-b7785c47de1f None None] MFA Rules not processed for user `15e3909f227543819e8c0ba513b5a41a`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:59:32.668808 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-48bb7910-b541-4161-9bd6-b7785c47de1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:32.734005 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1269/2546] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:59:32 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:59:32.738884 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ece72-52a1-4a6f-ba2a-b4bf0a95da07 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:32.780597 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:59:32.803171 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ece72-52a1-4a6f-ba2a-b4bf0a95da07 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:32.809117 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b880cca4-7bd8-425d-8740-65f60db5e30f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:32.809789 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b880cca4-7bd8-425d-8740-65f60db5e30f admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:32.810075 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b880cca4-7bd8-425d-8740-65f60db5e30f admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:32.810351 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b880cca4-7bd8-425d-8740-65f60db5e30f admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:32.811035 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b880cca4-7bd8-425d-8740-65f60db5e30f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:32.812988 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b880cca4-7bd8-425d-8740-65f60db5e30f admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:32.913577 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1278/2547] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:59:32 2022] POST /identity/v3/projects => generated 377 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:59:32.917831 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bb7910-b541-4161-9bd6-b7785c47de1f None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:32.928854 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bb7910-b541-4161-9bd6-b7785c47de1f None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:32.986095 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9959f190-a4e2-411e-a61f-ef6c9f421c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:32.986756 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9959f190-a4e2-411e-a61f-ef6c9f421c26 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:32.987047 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9959f190-a4e2-411e-a61f-ef6c9f421c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:32.987303 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9959f190-a4e2-411e-a61f-ef6c9f421c26 admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:32.987994 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9959f190-a4e2-411e-a61f-ef6c9f421c26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:32.989939 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9959f190-a4e2-411e-a61f-ef6c9f421c26 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:33.111722 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1270/2548] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:59:32 2022] POST /identity/v3/users => generated 289 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:59:33.115859 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b880cca4-7bd8-425d-8740-65f60db5e30f admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:33.126489 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b880cca4-7bd8-425d-8740-65f60db5e30f admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:33.131440 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc4ade0-5ea0-4edc-a0d6-377fd7fb8a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:33.132129 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc4ade0-5ea0-4edc-a0d6-377fd7fb8a55 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:33.132408 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc4ade0-5ea0-4edc-a0d6-377fd7fb8a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:33.132688 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc4ade0-5ea0-4edc-a0d6-377fd7fb8a55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:33.133418 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc4ade0-5ea0-4edc-a0d6-377fd7fb8a55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:33.135015 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc4ade0-5ea0-4edc-a0d6-377fd7fb8a55 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:33.145890 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1279/2549] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:59:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:59:33.152826 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9959f190-a4e2-411e-a61f-ef6c9f421c26 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:33.164185 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9959f190-a4e2-411e-a61f-ef6c9f421c26 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:33.168978 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e0de5f-39f6-409e-a9a8-61126a1d8026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:33.169576 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e0de5f-39f6-409e-a9a8-61126a1d8026 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:33.169862 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e0de5f-39f6-409e-a9a8-61126a1d8026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:33.170140 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e0de5f-39f6-409e-a9a8-61126a1d8026 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:33.170862 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e0de5f-39f6-409e-a9a8-61126a1d8026 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:33.172503 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e0de5f-39f6-409e-a9a8-61126a1d8026 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:33.178228 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:59:33.183720 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1271/2550] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:59:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:59:33.193287 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc4ade0-5ea0-4edc-a0d6-377fd7fb8a55 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:33.203604 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc4ade0-5ea0-4edc-a0d6-377fd7fb8a55 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:33.208330 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5563076-c154-469d-bbad-f65e83f2cace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:33.209186 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5563076-c154-469d-bbad-f65e83f2cace admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:33.209469 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5563076-c154-469d-bbad-f65e83f2cace admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:33.210443 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5563076-c154-469d-bbad-f65e83f2cace admin admin] PATH_INFO: `/v3/projects/b3fe4fb70a164ed2a00e4f19c886b73c/users/6d72f584222340d29a332bf00f2d006f/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:33.227982 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5563076-c154-469d-bbad-f65e83f2cace admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3fe4fb70a164ed2a00e4f19c886b73c, user_id=6d72f584222340d29a332bf00f2d006f, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:33.230671 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5563076-c154-469d-bbad-f65e83f2cace admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:33.329036 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1280/2551] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:59:33 2022] PUT /identity/v3/projects/b3fe4fb70a164ed2a00e4f19c886b73c/users/6d72f584222340d29a332bf00f2d006f/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:59:33.340892 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca68af6-6170-449b-827d-e80de89958f3 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:33.341174 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca68af6-6170-449b-827d-e80de89958f3 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:33.341455 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca68af6-6170-449b-827d-e80de89958f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:33.369374 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-dca68af6-6170-449b-827d-e80de89958f3 None None] MFA Rules not processed for user `6d72f584222340d29a332bf00f2d006f`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:59:33.370687 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-dca68af6-6170-449b-827d-e80de89958f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:33.436725 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1272/2552] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:59:33 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:59:33.441149 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5563076-c154-469d-bbad-f65e83f2cace admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:33.506541 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5563076-c154-469d-bbad-f65e83f2cace admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:33.512127 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b124f73f-685d-41a0-bd8f-838bfa583f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:33.512786 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b124f73f-685d-41a0-bd8f-838bfa583f5c admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:33.513070 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b124f73f-685d-41a0-bd8f-838bfa583f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:33.513349 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b124f73f-685d-41a0-bd8f-838bfa583f5c admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:33.514050 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b124f73f-685d-41a0-bd8f-838bfa583f5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:33.516049 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b124f73f-685d-41a0-bd8f-838bfa583f5c admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:33.621254 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1281/2553] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:59:33 2022] POST /identity/v3/projects => generated 375 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:59:33.625342 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca68af6-6170-449b-827d-e80de89958f3 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:33.637257 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca68af6-6170-449b-827d-e80de89958f3 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:33.693326 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b0f6d9-7c66-406b-986e-5d7ddb0e8acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:33.693911 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b0f6d9-7c66-406b-986e-5d7ddb0e8acf admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:33.694190 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b0f6d9-7c66-406b-986e-5d7ddb0e8acf admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:33.694467 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b0f6d9-7c66-406b-986e-5d7ddb0e8acf admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:33.696035 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b0f6d9-7c66-406b-986e-5d7ddb0e8acf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:33.697961 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b0f6d9-7c66-406b-986e-5d7ddb0e8acf admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:33.847305 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1273/2554] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:59:33 2022] POST /identity/v3/users => generated 294 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:59:33.852148 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b124f73f-685d-41a0-bd8f-838bfa583f5c admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:33.862280 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b124f73f-685d-41a0-bd8f-838bfa583f5c admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:33.867623 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9049c8-4076-401d-bdb9-e93bd9b3d84d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:33.868336 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9049c8-4076-401d-bdb9-e93bd9b3d84d admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:33.868620 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9049c8-4076-401d-bdb9-e93bd9b3d84d admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:33.868897 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9049c8-4076-401d-bdb9-e93bd9b3d84d admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:33.869633 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9049c8-4076-401d-bdb9-e93bd9b3d84d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:33.871394 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9049c8-4076-401d-bdb9-e93bd9b3d84d admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:33.883523 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1282/2555] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:59:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:59:33.887981 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b0f6d9-7c66-406b-986e-5d7ddb0e8acf admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:33.898151 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b0f6d9-7c66-406b-986e-5d7ddb0e8acf admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:33.902660 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277cbff5-8902-4ce4-8150-b63d63c8007c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:33.903528 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277cbff5-8902-4ce4-8150-b63d63c8007c admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:33.903875 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277cbff5-8902-4ce4-8150-b63d63c8007c admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:33.904191 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277cbff5-8902-4ce4-8150-b63d63c8007c admin admin] PATH_INFO: `/v3/projects/9ded08bb95ef49d7828969c367fb8171/users/a8a3861ffacd4d718495c3a87dcd7e8c/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:33.919376 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277cbff5-8902-4ce4-8150-b63d63c8007c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ded08bb95ef49d7828969c367fb8171, user_id=a8a3861ffacd4d718495c3a87dcd7e8c, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:33.922180 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277cbff5-8902-4ce4-8150-b63d63c8007c admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:34.030036 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1274/2556] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:59:33 2022] PUT /identity/v3/projects/9ded08bb95ef49d7828969c367fb8171/users/a8a3861ffacd4d718495c3a87dcd7e8c/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:59:34.034136 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9049c8-4076-401d-bdb9-e93bd9b3d84d admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:34.044847 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9049c8-4076-401d-bdb9-e93bd9b3d84d admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:34.100430 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea992f89-af35-4e3d-b99a-6ac4ffea1879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:34.101617 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea992f89-af35-4e3d-b99a-6ac4ffea1879 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:34.101901 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea992f89-af35-4e3d-b99a-6ac4ffea1879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:34.102368 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea992f89-af35-4e3d-b99a-6ac4ffea1879 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:34.103092 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea992f89-af35-4e3d-b99a-6ac4ffea1879 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:34.104844 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea992f89-af35-4e3d-b99a-6ac4ffea1879 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:34.106113 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:59:34.115692 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1283/2557] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:59:34 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:59:34.120991 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277cbff5-8902-4ce4-8150-b63d63c8007c admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:34.131809 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277cbff5-8902-4ce4-8150-b63d63c8007c admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:34.136990 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117e5e1d-77d0-4890-9d6c-adf22956fcbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:34.137901 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e5e1d-77d0-4890-9d6c-adf22956fcbd admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:34.138184 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e5e1d-77d0-4890-9d6c-adf22956fcbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:34.138462 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e5e1d-77d0-4890-9d6c-adf22956fcbd admin admin] PATH_INFO: `/v3/projects/9ded08bb95ef49d7828969c367fb8171/users/a8a3861ffacd4d718495c3a87dcd7e8c/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:34.140515 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117e5e1d-77d0-4890-9d6c-adf22956fcbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ded08bb95ef49d7828969c367fb8171, user_id=a8a3861ffacd4d718495c3a87dcd7e8c, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:34.143158 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117e5e1d-77d0-4890-9d6c-adf22956fcbd admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:34.154810 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1275/2558] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:59:34 2022] PUT /identity/v3/projects/9ded08bb95ef49d7828969c367fb8171/users/a8a3861ffacd4d718495c3a87dcd7e8c/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:59:34.165885 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21dc631-79b4-42da-952c-828c883e1faa None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:34.166170 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21dc631-79b4-42da-952c-828c883e1faa None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:34.166445 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21dc631-79b4-42da-952c-828c883e1faa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:34.194898 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-a21dc631-79b4-42da-952c-828c883e1faa None None] MFA Rules not processed for user `a8a3861ffacd4d718495c3a87dcd7e8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:59:34.196109 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a21dc631-79b4-42da-952c-828c883e1faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:34.264758 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1284/2559] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:59:34 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:59:34.276991 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117e5e1d-77d0-4890-9d6c-adf22956fcbd admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:34.278717 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-117e5e1d-77d0-4890-9d6c-adf22956fcbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:34.325565 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:59:34.337696 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117e5e1d-77d0-4890-9d6c-adf22956fcbd admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:34.341452 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20748980-ea58-4e4f-8844-893628440a00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:34.342097 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20748980-ea58-4e4f-8844-893628440a00 service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:34.342378 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20748980-ea58-4e4f-8844-893628440a00 service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:34.342698 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20748980-ea58-4e4f-8844-893628440a00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:34.401377 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20748980-ea58-4e4f-8844-893628440a00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:34.402935 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20748980-ea58-4e4f-8844-893628440a00 service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:34.412330 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1276/2560] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:59:34 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:59:37.489489 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21dc631-79b4-42da-952c-828c883e1faa None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:37.499589 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21dc631-79b4-42da-952c-828c883e1faa None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:37.503366 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7f7764-d242-4aca-a2f7-2fa9c50cc565 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:37.503995 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7f7764-d242-4aca-a2f7-2fa9c50cc565 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:37.504374 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7f7764-d242-4aca-a2f7-2fa9c50cc565 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:37.504653 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7f7764-d242-4aca-a2f7-2fa9c50cc565 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:37.562844 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7f7764-d242-4aca-a2f7-2fa9c50cc565 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:37.564317 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7f7764-d242-4aca-a2f7-2fa9c50cc565 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:37.573902 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1285/2561] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:59:37 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:59:39.166447 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20748980-ea58-4e4f-8844-893628440a00 service cinder] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:39.174843 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20748980-ea58-4e4f-8844-893628440a00 service cinder] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:39.178394 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5e1e45-c560-4d0e-8a8b-2b2a870c3ea3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:39.179021 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e1e45-c560-4d0e-8a8b-2b2a870c3ea3 service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:39.179303 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e1e45-c560-4d0e-8a8b-2b2a870c3ea3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:39.179582 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e1e45-c560-4d0e-8a8b-2b2a870c3ea3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:39.189300 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5e1e45-c560-4d0e-8a8b-2b2a870c3ea3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:39.190708 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5e1e45-c560-4d0e-8a8b-2b2a870c3ea3 service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:39.200075 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1277/2562] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:59:39 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:59:44.944651 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7f7764-d242-4aca-a2f7-2fa9c50cc565 service cinder] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:45.006637 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7f7764-d242-4aca-a2f7-2fa9c50cc565 service cinder] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:45.011212 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808b7c38-5cf1-4638-b03e-d8ff9947f473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:45.012171 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808b7c38-5cf1-4638-b03e-d8ff9947f473 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:45.012413 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808b7c38-5cf1-4638-b03e-d8ff9947f473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:45.012696 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808b7c38-5cf1-4638-b03e-d8ff9947f473 admin admin] PATH_INFO: `/v3/users/15e3909f227543819e8c0ba513b5a41a` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:45.013882 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808b7c38-5cf1-4638-b03e-d8ff9947f473 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15e3909f227543819e8c0ba513b5a41a)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:45.015825 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808b7c38-5cf1-4638-b03e-d8ff9947f473 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:45.037796 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:59:45.234380 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-808b7c38-5cf1-4638-b03e-d8ff9947f473 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15e3909f227543819e8c0ba513b5a41a'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:45.235131 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-808b7c38-5cf1-4638-b03e-d8ff9947f473 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15e3909f227543819e8c0ba513b5a41a'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:45.235738 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-808b7c38-5cf1-4638-b03e-d8ff9947f473 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15e3909f227543819e8c0ba513b5a41a'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:45.304949 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-808b7c38-5cf1-4638-b03e-d8ff9947f473 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15e3909f227543819e8c0ba513b5a41a'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:45.318626 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-808b7c38-5cf1-4638-b03e-d8ff9947f473 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15e3909f227543819e8c0ba513b5a41a'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:45.332032 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1286/2563] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:59:44 2022] DELETE /identity/v3/users/15e3909f227543819e8c0ba513b5a41a => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:59:45.336595 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800b1e5c-d9ed-4db2-b847-f628596eed5f None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:45.337087 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800b1e5c-d9ed-4db2-b847-f628596eed5f None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:45.337191 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800b1e5c-d9ed-4db2-b847-f628596eed5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:45.355759 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-800b1e5c-d9ed-4db2-b847-f628596eed5f None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:59:45.360419 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-800b1e5c-d9ed-4db2-b847-f628596eed5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:45.419625 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1278/2564] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:59:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:59:45.430295 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808b7c38-5cf1-4638-b03e-d8ff9947f473 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:45.431697 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-808b7c38-5cf1-4638-b03e-d8ff9947f473 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:45.488815 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808b7c38-5cf1-4638-b03e-d8ff9947f473 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:45.492915 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8a76c2-4098-4bd7-9033-533d37783f9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:45.492915 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8a76c2-4098-4bd7-9033-533d37783f9e service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:45.493192 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8a76c2-4098-4bd7-9033-533d37783f9e service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:45.493469 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8a76c2-4098-4bd7-9033-533d37783f9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:45.504190 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8a76c2-4098-4bd7-9033-533d37783f9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:45.505614 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8a76c2-4098-4bd7-9033-533d37783f9e service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:45.515404 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1287/2565] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:59:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:59:45.941151 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800b1e5c-d9ed-4db2-b847-f628596eed5f None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:45.942600 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-800b1e5c-d9ed-4db2-b847-f628596eed5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:45.955685 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800b1e5c-d9ed-4db2-b847-f628596eed5f None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:45.960336 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb0458e-c09b-46a7-b2d6-561d68b446ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:45.961336 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb0458e-c09b-46a7-b2d6-561d68b446ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:45.961620 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb0458e-c09b-46a7-b2d6-561d68b446ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:45.961897 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb0458e-c09b-46a7-b2d6-561d68b446ac admin admin] PATH_INFO: `/v3/projects/15e2a5209f0541008624c0e91e4c5866` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:45.963065 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb0458e-c09b-46a7-b2d6-561d68b446ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=15e2a5209f0541008624c0e91e4c5866)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:45.965840 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb0458e-c09b-46a7-b2d6-561d68b446ac admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:46.070881 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-cbb0458e-c09b-46a7-b2d6-561d68b446ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15e2a5209f0541008624c0e91e4c5866', 'request_id': 'req-cbb0458e-c09b-46a7-b2d6-561d68b446ac'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:46.162498 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-cbb0458e-c09b-46a7-b2d6-561d68b446ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15e2a5209f0541008624c0e91e4c5866', 'request_id': 'req-cbb0458e-c09b-46a7-b2d6-561d68b446ac'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:46.168863 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-cbb0458e-c09b-46a7-b2d6-561d68b446ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15e2a5209f0541008624c0e91e4c5866', 'request_id': 'req-cbb0458e-c09b-46a7-b2d6-561d68b446ac'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:46.179973 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-cbb0458e-c09b-46a7-b2d6-561d68b446ac admin admin] The token cache is being invalidate because project 15e2a5209f0541008624c0e91e4c5866 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:59:46.180289 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-cbb0458e-c09b-46a7-b2d6-561d68b446ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:46.180972 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-cbb0458e-c09b-46a7-b2d6-561d68b446ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:46.182534 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1279/2566] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:59:45 2022] DELETE /identity/v3/projects/15e2a5209f0541008624c0e91e4c5866 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:59:46.186608 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8a76c2-4098-4bd7-9033-533d37783f9e service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:46.196413 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8a76c2-4098-4bd7-9033-533d37783f9e service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:46.255322 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4ec782-0413-4a02-b823-f6ed6607199b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:46.256394 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4ec782-0413-4a02-b823-f6ed6607199b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:46.256519 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4ec782-0413-4a02-b823-f6ed6607199b admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:46.256746 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4ec782-0413-4a02-b823-f6ed6607199b admin admin] PATH_INFO: `/v3/users/6d72f584222340d29a332bf00f2d006f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:46.258008 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4ec782-0413-4a02-b823-f6ed6607199b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d72f584222340d29a332bf00f2d006f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:46.259967 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4ec782-0413-4a02-b823-f6ed6607199b admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:46.438596 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:59:46.441261 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-bb4ec782-0413-4a02-b823-f6ed6607199b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d72f584222340d29a332bf00f2d006f'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:46.441899 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-bb4ec782-0413-4a02-b823-f6ed6607199b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d72f584222340d29a332bf00f2d006f'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:46.442645 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-bb4ec782-0413-4a02-b823-f6ed6607199b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d72f584222340d29a332bf00f2d006f'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:46.572558 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-bb4ec782-0413-4a02-b823-f6ed6607199b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d72f584222340d29a332bf00f2d006f'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:46.584772 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-bb4ec782-0413-4a02-b823-f6ed6607199b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d72f584222340d29a332bf00f2d006f'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:46.597547 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1288/2567] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:59:46 2022] DELETE /identity/v3/users/6d72f584222340d29a332bf00f2d006f => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:59:47.044180 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb0458e-c09b-46a7-b2d6-561d68b446ac admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:47.045774 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-cbb0458e-c09b-46a7-b2d6-561d68b446ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:47.064797 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb0458e-c09b-46a7-b2d6-561d68b446ac admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:47.069449 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:59:47.123030 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b677593-da65-40b7-b5f2-de5ab445a19a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:47.123896 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b677593-da65-40b7-b5f2-de5ab445a19a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:47.124207 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b677593-da65-40b7-b5f2-de5ab445a19a admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:47.124483 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b677593-da65-40b7-b5f2-de5ab445a19a admin admin] PATH_INFO: `/v3/projects/b3fe4fb70a164ed2a00e4f19c886b73c` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:47.125682 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b677593-da65-40b7-b5f2-de5ab445a19a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3fe4fb70a164ed2a00e4f19c886b73c)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:47.127811 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b677593-da65-40b7-b5f2-de5ab445a19a admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:47.242341 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-7b677593-da65-40b7-b5f2-de5ab445a19a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3fe4fb70a164ed2a00e4f19c886b73c', 'request_id': 'req-7b677593-da65-40b7-b5f2-de5ab445a19a'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:47.328274 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-7b677593-da65-40b7-b5f2-de5ab445a19a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3fe4fb70a164ed2a00e4f19c886b73c', 'request_id': 'req-7b677593-da65-40b7-b5f2-de5ab445a19a'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:47.334351 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-7b677593-da65-40b7-b5f2-de5ab445a19a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3fe4fb70a164ed2a00e4f19c886b73c', 'request_id': 'req-7b677593-da65-40b7-b5f2-de5ab445a19a'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:47.345874 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-7b677593-da65-40b7-b5f2-de5ab445a19a admin admin] The token cache is being invalidate because project b3fe4fb70a164ed2a00e4f19c886b73c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:59:47.346161 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-7b677593-da65-40b7-b5f2-de5ab445a19a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:47.346808 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-7b677593-da65-40b7-b5f2-de5ab445a19a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:47.348404 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1280/2568] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:59:47 2022] DELETE /identity/v3/projects/b3fe4fb70a164ed2a00e4f19c886b73c => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:59:47.352457 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4ec782-0413-4a02-b823-f6ed6607199b admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:47.353975 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-bb4ec782-0413-4a02-b823-f6ed6607199b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:47.366896 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4ec782-0413-4a02-b823-f6ed6607199b admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:47.425284 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aae83a4-3921-47e3-a99e-ab39f669d1a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:47.426186 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aae83a4-3921-47e3-a99e-ab39f669d1a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:47.426470 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aae83a4-3921-47e3-a99e-ab39f669d1a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:47.426748 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aae83a4-3921-47e3-a99e-ab39f669d1a4 admin admin] PATH_INFO: `/v3/users/a8a3861ffacd4d718495c3a87dcd7e8c` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:47.427932 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aae83a4-3921-47e3-a99e-ab39f669d1a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8a3861ffacd4d718495c3a87dcd7e8c)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:47.429902 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aae83a4-3921-47e3-a99e-ab39f669d1a4 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:47.651575 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-1aae83a4-3921-47e3-a99e-ab39f669d1a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8a3861ffacd4d718495c3a87dcd7e8c'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:47.652184 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-1aae83a4-3921-47e3-a99e-ab39f669d1a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8a3861ffacd4d718495c3a87dcd7e8c'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:47.652804 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-1aae83a4-3921-47e3-a99e-ab39f669d1a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8a3861ffacd4d718495c3a87dcd7e8c'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:47.738205 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-1aae83a4-3921-47e3-a99e-ab39f669d1a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8a3861ffacd4d718495c3a87dcd7e8c'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:47.751232 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-1aae83a4-3921-47e3-a99e-ab39f669d1a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8a3861ffacd4d718495c3a87dcd7e8c'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:47.764130 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1289/2569] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 14:59:47 2022] DELETE /identity/v3/users/a8a3861ffacd4d718495c3a87dcd7e8c => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:59:48.198126 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b677593-da65-40b7-b5f2-de5ab445a19a admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:48.200542 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-7b677593-da65-40b7-b5f2-de5ab445a19a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:48.219362 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b677593-da65-40b7-b5f2-de5ab445a19a admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:48.279372 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c37706-c1f0-494e-bbac-f7b3364d99c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:48.280273 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c37706-c1f0-494e-bbac-f7b3364d99c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:48.280553 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c37706-c1f0-494e-bbac-f7b3364d99c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:48.280825 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c37706-c1f0-494e-bbac-f7b3364d99c1 admin admin] PATH_INFO: `/v3/projects/9ded08bb95ef49d7828969c367fb8171` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:48.282007 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c37706-c1f0-494e-bbac-f7b3364d99c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ded08bb95ef49d7828969c367fb8171)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:48.283885 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c37706-c1f0-494e-bbac-f7b3364d99c1 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:48.390378 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:59:48.397021 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-84c37706-c1f0-494e-bbac-f7b3364d99c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ded08bb95ef49d7828969c367fb8171', 'request_id': 'req-84c37706-c1f0-494e-bbac-f7b3364d99c1'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:48.463533 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-84c37706-c1f0-494e-bbac-f7b3364d99c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ded08bb95ef49d7828969c367fb8171', 'request_id': 'req-84c37706-c1f0-494e-bbac-f7b3364d99c1'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:48.469999 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-84c37706-c1f0-494e-bbac-f7b3364d99c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ded08bb95ef49d7828969c367fb8171', 'request_id': 'req-84c37706-c1f0-494e-bbac-f7b3364d99c1'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:48.481621 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-84c37706-c1f0-494e-bbac-f7b3364d99c1 admin admin] The token cache is being invalidate because project 9ded08bb95ef49d7828969c367fb8171 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:59:48.481910 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-84c37706-c1f0-494e-bbac-f7b3364d99c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:48.482574 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-84c37706-c1f0-494e-bbac-f7b3364d99c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:48.484137 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1281/2570] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 14:59:48 2022] DELETE /identity/v3/projects/9ded08bb95ef49d7828969c367fb8171 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:59:48.489023 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c08f9c-f5a2-42fd-9fde-ed4ce4c50f03 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:48.489503 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c08f9c-f5a2-42fd-9fde-ed4ce4c50f03 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:48.489609 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c08f9c-f5a2-42fd-9fde-ed4ce4c50f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:48.509031 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-90c08f9c-f5a2-42fd-9fde-ed4ce4c50f03 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:59:48.510281 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-90c08f9c-f5a2-42fd-9fde-ed4ce4c50f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:48.547065 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:59:48.573419 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1290/2571] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:59:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:59:48.583020 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c9741a-6e4c-4c0d-a6e3-c097d9d0308f None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:48.583407 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c9741a-6e4c-4c0d-a6e3-c097d9d0308f None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:48.583800 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c9741a-6e4c-4c0d-a6e3-c097d9d0308f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:48.602394 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-09c9741a-6e4c-4c0d-a6e3-c097d9d0308f None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:59:48.603601 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-09c9741a-6e4c-4c0d-a6e3-c097d9d0308f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:48.612015 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1282/2572] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:59:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:59:48.616655 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c08f9c-f5a2-42fd-9fde-ed4ce4c50f03 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:48.628352 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c08f9c-f5a2-42fd-9fde-ed4ce4c50f03 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:48.633044 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13553646-da7a-4855-ab6f-dafd7a081b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:48.633910 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13553646-da7a-4855-ab6f-dafd7a081b12 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:48.634197 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13553646-da7a-4855-ab6f-dafd7a081b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:48.634509 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13553646-da7a-4855-ab6f-dafd7a081b12 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:48.635260 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13553646-da7a-4855-ab6f-dafd7a081b12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:59:48.635591 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13553646-da7a-4855-ab6f-dafd7a081b12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:48.638209 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13553646-da7a-4855-ab6f-dafd7a081b12 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:48.651544 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1291/2573] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jun 15 14:59:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:59:48.656257 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c9741a-6e4c-4c0d-a6e3-c097d9d0308f None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:48.665613 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c9741a-6e4c-4c0d-a6e3-c097d9d0308f None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:48.670879 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b55139b-58e0-442c-a71e-822828415adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:48.671531 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b55139b-58e0-442c-a71e-822828415adf admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:48.679581 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b55139b-58e0-442c-a71e-822828415adf admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:48.679581 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b55139b-58e0-442c-a71e-822828415adf admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:48.679581 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b55139b-58e0-442c-a71e-822828415adf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:48.679581 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b55139b-58e0-442c-a71e-822828415adf admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:48.770452 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1283/2574] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 14:59:48 2022] POST /identity/v3/projects => generated 373 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:59:48.774543 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13553646-da7a-4855-ab6f-dafd7a081b12 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:48.784759 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13553646-da7a-4855-ab6f-dafd7a081b12 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:48.843282 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c1f090-7e8c-4b63-88b2-14373cb5c754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:48.843963 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c1f090-7e8c-4b63-88b2-14373cb5c754 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:48.844298 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c1f090-7e8c-4b63-88b2-14373cb5c754 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:48.844580 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c1f090-7e8c-4b63-88b2-14373cb5c754 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:48.845262 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c1f090-7e8c-4b63-88b2-14373cb5c754 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:48.847161 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c1f090-7e8c-4b63-88b2-14373cb5c754 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:48.957031 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1292/2575] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 14:59:48 2022] POST /identity/v3/users => generated 287 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:59:48.961260 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b55139b-58e0-442c-a71e-822828415adf admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:48.971430 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b55139b-58e0-442c-a71e-822828415adf admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:48.976109 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f2ce25-23b9-44d9-a82d-797906b0a01a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:48.976760 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f2ce25-23b9-44d9-a82d-797906b0a01a admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:48.977042 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f2ce25-23b9-44d9-a82d-797906b0a01a admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:48.977320 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f2ce25-23b9-44d9-a82d-797906b0a01a admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:48.978093 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f2ce25-23b9-44d9-a82d-797906b0a01a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:48.979682 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f2ce25-23b9-44d9-a82d-797906b0a01a admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:48.990596 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1284/2576] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:59:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:59:48.994650 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c1f090-7e8c-4b63-88b2-14373cb5c754 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:49.006035 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c1f090-7e8c-4b63-88b2-14373cb5c754 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:49.010739 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf4421-a425-4ce0-9a06-113f4ac6e68f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:49.011386 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf4421-a425-4ce0-9a06-113f4ac6e68f admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:49.011787 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf4421-a425-4ce0-9a06-113f4ac6e68f admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:49.012300 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf4421-a425-4ce0-9a06-113f4ac6e68f admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:49.013118 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdf4421-a425-4ce0-9a06-113f4ac6e68f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:49.014726 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdf4421-a425-4ce0-9a06-113f4ac6e68f admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:49.025236 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1293/2577] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 14:59:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:59:49.029649 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f2ce25-23b9-44d9-a82d-797906b0a01a admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:49.039739 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f2ce25-23b9-44d9-a82d-797906b0a01a admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:49.045305 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b96ea75-d282-4741-bcf2-839779c2b576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:49.046176 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b96ea75-d282-4741-bcf2-839779c2b576 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:49.046528 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b96ea75-d282-4741-bcf2-839779c2b576 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:49.046810 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b96ea75-d282-4741-bcf2-839779c2b576 admin admin] PATH_INFO: `/v3/projects/c0a0e27741434fd282a638aefa713c91/users/33f42c17c1374253b7716ffe0ffbaa6d/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:49.062129 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b96ea75-d282-4741-bcf2-839779c2b576 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0a0e27741434fd282a638aefa713c91, user_id=33f42c17c1374253b7716ffe0ffbaa6d, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:49.065171 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b96ea75-d282-4741-bcf2-839779c2b576 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:49.191792 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1285/2578] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 14:59:49 2022] PUT /identity/v3/projects/c0a0e27741434fd282a638aefa713c91/users/33f42c17c1374253b7716ffe0ffbaa6d/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:59:49.196998 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaf3b3d-cd64-449a-89f1-bba6e5143600 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:49.197293 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaf3b3d-cd64-449a-89f1-bba6e5143600 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:49.199024 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaf3b3d-cd64-449a-89f1-bba6e5143600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:49.216795 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-deaf3b3d-cd64-449a-89f1-bba6e5143600 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:59:49.218122 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-deaf3b3d-cd64-449a-89f1-bba6e5143600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:49.283220 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1294/2579] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:59:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:59:49.292755 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b96ea75-d282-4741-bcf2-839779c2b576 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:49.295087 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-2b96ea75-d282-4741-bcf2-839779c2b576 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:49.350787 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b96ea75-d282-4741-bcf2-839779c2b576 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:49.354371 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e690ba-b95a-48cb-845a-1054f19e5048 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:49.355003 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e690ba-b95a-48cb-845a-1054f19e5048 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:49.355282 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e690ba-b95a-48cb-845a-1054f19e5048 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:49.356139 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e690ba-b95a-48cb-845a-1054f19e5048 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:49.365310 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e690ba-b95a-48cb-845a-1054f19e5048 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:49.366721 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e690ba-b95a-48cb-845a-1054f19e5048 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:49.376605 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1286/2580] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:59:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:59:50.637694 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23792e2e-74a2-4f17-a6c4-d7fb0f77716a None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:50.638217 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23792e2e-74a2-4f17-a6c4-d7fb0f77716a None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:50.638217 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23792e2e-74a2-4f17-a6c4-d7fb0f77716a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:50.667009 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-23792e2e-74a2-4f17-a6c4-d7fb0f77716a None None] MFA Rules not processed for user `33f42c17c1374253b7716ffe0ffbaa6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:59:50.668391 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-23792e2e-74a2-4f17-a6c4-d7fb0f77716a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:50.705243 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:59:50.739584 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1295/2581] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:59:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:59:50.753356 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e690ba-b95a-48cb-845a-1054f19e5048 service neutron] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:50.755121 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-70e690ba-b95a-48cb-845a-1054f19e5048 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:50.815331 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e690ba-b95a-48cb-845a-1054f19e5048 service neutron] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:50.818965 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b538b1e-b968-49df-b020-874fe68523e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:50.819608 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b538b1e-b968-49df-b020-874fe68523e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:50.819894 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b538b1e-b968-49df-b020-874fe68523e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:50.820203 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b538b1e-b968-49df-b020-874fe68523e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:50.828673 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b538b1e-b968-49df-b020-874fe68523e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:50.830105 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b538b1e-b968-49df-b020-874fe68523e9 service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:50.839084 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1287/2582] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 14:59:50 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:59:53.542618 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385744d7-f80e-432e-8f18-e594037a434d None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:53.543169 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385744d7-f80e-432e-8f18-e594037a434d None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:53.543169 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385744d7-f80e-432e-8f18-e594037a434d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:53.563191 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-385744d7-f80e-432e-8f18-e594037a434d None None] MFA Rules not processed for user `33f42c17c1374253b7716ffe0ffbaa6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:59:53.564551 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-385744d7-f80e-432e-8f18-e594037a434d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:53.571680 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1296/2583] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 14:59:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:00:00.835050 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=89,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:00:03.557241 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=14 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:00:12.062943 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b538b1e-b968-49df-b020-874fe68523e9 service cinder] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:00:12.072444 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b538b1e-b968-49df-b020-874fe68523e9 service cinder] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:00:12.076063 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2caf0e8-6f0b-483e-88c2-1e0e1ae0a230 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:00:12.076744 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2caf0e8-6f0b-483e-88c2-1e0e1ae0a230 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:00:12.077072 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2caf0e8-6f0b-483e-88c2-1e0e1ae0a230 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:00:12.077368 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2caf0e8-6f0b-483e-88c2-1e0e1ae0a230 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:00:12.079311 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-f2caf0e8-6f0b-483e-88c2-1e0e1ae0a230 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:00:12.154488 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2caf0e8-6f0b-483e-88c2-1e0e1ae0a230 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:00:12.156022 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2caf0e8-6f0b-483e-88c2-1e0e1ae0a230 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:00:12.167259 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1288/2584] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:00:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:00:13.692385 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385744d7-f80e-432e-8f18-e594037a434d None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:00:13.702233 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385744d7-f80e-432e-8f18-e594037a434d None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:00:13.705801 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a7d7c7-c9e3-44ca-8da3-e27c66b0be95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:00:13.706429 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a7d7c7-c9e3-44ca-8da3-e27c66b0be95 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:00:13.706709 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a7d7c7-c9e3-44ca-8da3-e27c66b0be95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:00:13.706986 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a7d7c7-c9e3-44ca-8da3-e27c66b0be95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:00:13.708758 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c8a7d7c7-c9e3-44ca-8da3-e27c66b0be95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:00:13.722615 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a7d7c7-c9e3-44ca-8da3-e27c66b0be95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:00:13.724878 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a7d7c7-c9e3-44ca-8da3-e27c66b0be95 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:00:13.737384 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1297/2585] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:00:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:00:16.094544 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb51a0-f799-4a33-b810-70d2ca20f0de None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:00:16.095183 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb51a0-f799-4a33-b810-70d2ca20f0de None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:00:16.095183 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb51a0-f799-4a33-b810-70d2ca20f0de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:00:16.113758 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-5fbb51a0-f799-4a33-b810-70d2ca20f0de None None] MFA Rules not processed for user `33f42c17c1374253b7716ffe0ffbaa6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:00:16.115687 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-5fbb51a0-f799-4a33-b810-70d2ca20f0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:00:16.122199 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1289/2586] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:00:16 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:00:23.731129 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=6 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:00:26.109079 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=37 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:00:45.797025 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d21b0ed-cfbe-4da7-9bf4-a996a3a20dae None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:00:45.804221 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d21b0ed-cfbe-4da7-9bf4-a996a3a20dae None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:00:45.804221 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d21b0ed-cfbe-4da7-9bf4-a996a3a20dae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:00:45.815919 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-8d21b0ed-cfbe-4da7-9bf4-a996a3a20dae None None] MFA Rules not processed for user `19d82c04f9994654adfb10f4bd0573c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:00:45.817191 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-8d21b0ed-cfbe-4da7-9bf4-a996a3a20dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:00:45.873228 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1298/2587] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Jun 15 15:00:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:00:46.028282 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbb51a0-f799-4a33-b810-70d2ca20f0de None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:00:46.029748 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-5fbb51a0-f799-4a33-b810-70d2ca20f0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:00:46.106328 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbb51a0-f799-4a33-b810-70d2ca20f0de None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:00:46.111548 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce53f7c-0361-4d12-ba09-c8b42f8bc1fd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04dc5832333498e96a7e02bbfb9484c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:00:46.112574 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce53f7c-0361-4d12-ba09-c8b42f8bc1fd service swift] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:00:46.112862 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce53f7c-0361-4d12-ba09-c8b42f8bc1fd service swift] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:00:46.113145 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce53f7c-0361-4d12-ba09-c8b42f8bc1fd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:00:46.122065 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce53f7c-0361-4d12-ba09-c8b42f8bc1fd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:00:46.123653 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce53f7c-0361-4d12-ba09-c8b42f8bc1fd service swift] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:00:46.133924 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1290/2588] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Jun 15 15:00:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 15 15:00:46.563001 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d21b0ed-cfbe-4da7-9bf4-a996a3a20dae None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:00:46.575753 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d21b0ed-cfbe-4da7-9bf4-a996a3a20dae None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:00:46.580696 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0af99df-0083-4f02-b592-7c0476444f88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:00:46.581656 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0af99df-0083-4f02-b592-7c0476444f88 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:00:46.582152 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0af99df-0083-4f02-b592-7c0476444f88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:00:46.582641 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0af99df-0083-4f02-b592-7c0476444f88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:00:46.584812 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-f0af99df-0083-4f02-b592-7c0476444f88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:00:46.657153 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0af99df-0083-4f02-b592-7c0476444f88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:00:46.659293 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0af99df-0083-4f02-b592-7c0476444f88 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:00:46.675641 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1299/2589] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:00:46 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:00:48.085347 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce53f7c-0361-4d12-ba09-c8b42f8bc1fd service swift] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:00:48.086798 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-dce53f7c-0361-4d12-ba09-c8b42f8bc1fd service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:00:48.118379 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce53f7c-0361-4d12-ba09-c8b42f8bc1fd service swift] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:00:48.128177 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ab02ad-bfe5-4494-966d-3e78d89afb76 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0927c05c340c48e48b18754888aa9c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:00:48.129763 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab02ad-bfe5-4494-966d-3e78d89afb76 service nova] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:00:48.130560 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab02ad-bfe5-4494-966d-3e78d89afb76 service nova] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:00:48.130851 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab02ad-bfe5-4494-966d-3e78d89afb76 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:00:48.133302 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-96ab02ad-bfe5-4494-966d-3e78d89afb76 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:00:48.154293 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ab02ad-bfe5-4494-966d-3e78d89afb76 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:00:48.155919 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ab02ad-bfe5-4494-966d-3e78d89afb76 service nova] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:00:48.173165 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1291/2590] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:00:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:00:48.458925 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0af99df-0083-4f02-b592-7c0476444f88 service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:00:48.469520 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0af99df-0083-4f02-b592-7c0476444f88 service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:00:48.475652 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd422e0-a0af-4919-b784-9c9301e91104 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:00:48.477010 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd422e0-a0af-4919-b784-9c9301e91104 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:00:48.477636 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd422e0-a0af-4919-b784-9c9301e91104 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:00:48.478221 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd422e0-a0af-4919-b784-9c9301e91104 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:00:48.480410 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-afd422e0-a0af-4919-b784-9c9301e91104 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:00:48.493144 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd422e0-a0af-4919-b784-9c9301e91104 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:00:48.494898 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd422e0-a0af-4919-b784-9c9301e91104 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:00:48.506475 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1300/2591] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:00:48 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:00:52.359078 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ab02ad-bfe5-4494-966d-3e78d89afb76 service nova] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:00:52.379627 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ab02ad-bfe5-4494-966d-3e78d89afb76 service nova] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:00:52.388453 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8449f1-9bda-4429-8e31-03141585cdc1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0927c05c340c48e48b18754888aa9c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:00:52.389295 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8449f1-9bda-4429-8e31-03141585cdc1 service nova] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:00:52.390302 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8449f1-9bda-4429-8e31-03141585cdc1 service nova] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:00:52.390302 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8449f1-9bda-4429-8e31-03141585cdc1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:00:52.393916 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-ba8449f1-9bda-4429-8e31-03141585cdc1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:00:52.411211 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8449f1-9bda-4429-8e31-03141585cdc1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:00:52.413053 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8449f1-9bda-4429-8e31-03141585cdc1 service nova] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:00:52.426645 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1292/2592] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:00:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:00:55.675927 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd422e0-a0af-4919-b784-9c9301e91104 service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:00:55.677580 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-afd422e0-a0af-4919-b784-9c9301e91104 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:00:55.696959 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd422e0-a0af-4919-b784-9c9301e91104 service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:00:55.706037 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc7e7c0-9f71-4413-b036-feba409221f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:00:55.708171 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc7e7c0-9f71-4413-b036-feba409221f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:00:55.709303 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc7e7c0-9f71-4413-b036-feba409221f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:00:55.710228 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc7e7c0-9f71-4413-b036-feba409221f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:00:55.714000 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-3cc7e7c0-9f71-4413-b036-feba409221f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:00:55.739386 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc7e7c0-9f71-4413-b036-feba409221f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:00:55.742385 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc7e7c0-9f71-4413-b036-feba409221f7 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:00:55.774550 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1301/2593] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:00:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 15:01:01.174927 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a126c2-85f6-473f-b53f-1e31f990f2d3 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:01.176020 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a126c2-85f6-473f-b53f-1e31f990f2d3 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:01.176581 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a126c2-85f6-473f-b53f-1e31f990f2d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:01.213006 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-94a126c2-85f6-473f-b53f-1e31f990f2d3 None None] MFA Rules not processed for user `19d82c04f9994654adfb10f4bd0573c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:01:01.214606 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-94a126c2-85f6-473f-b53f-1e31f990f2d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:01.223919 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1293/2594] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Jun 15 15:01:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:01:01.254191 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc7e7c0-9f71-4413-b036-feba409221f7 service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:01.268773 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc7e7c0-9f71-4413-b036-feba409221f7 service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:01.276982 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764f106e-b670-4a4b-aee6-482acb7f2c39 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04dc5832333498e96a7e02bbfb9484c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:01.277720 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764f106e-b670-4a4b-aee6-482acb7f2c39 service swift] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:01.278081 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764f106e-b670-4a4b-aee6-482acb7f2c39 service swift] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:01.278431 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764f106e-b670-4a4b-aee6-482acb7f2c39 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:01.290673 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764f106e-b670-4a4b-aee6-482acb7f2c39 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:01.293236 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764f106e-b670-4a4b-aee6-482acb7f2c39 service swift] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:01.306019 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1302/2595] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Jun 15 15:01:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 15:01:08.657992 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a126c2-85f6-473f-b53f-1e31f990f2d3 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:08.660438 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-94a126c2-85f6-473f-b53f-1e31f990f2d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:08.678209 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=48 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:01:08.702002 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a126c2-85f6-473f-b53f-1e31f990f2d3 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:08.715156 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e03328-6b55-4316-bb24-6142647f9ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:08.716440 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e03328-6b55-4316-bb24-6142647f9ceb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:08.716992 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e03328-6b55-4316-bb24-6142647f9ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:08.717381 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e03328-6b55-4316-bb24-6142647f9ceb admin admin] PATH_INFO: `/v3/users/a0e682af1a7f493c992509142b673932` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:08.721250 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e03328-6b55-4316-bb24-6142647f9ceb admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0e682af1a7f493c992509142b673932)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:08.728757 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e03328-6b55-4316-bb24-6142647f9ceb admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:08.990783 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-77e03328-6b55-4316-bb24-6142647f9ceb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0e682af1a7f493c992509142b673932'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:08.992200 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-77e03328-6b55-4316-bb24-6142647f9ceb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0e682af1a7f493c992509142b673932'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:09.005842 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-77e03328-6b55-4316-bb24-6142647f9ceb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0e682af1a7f493c992509142b673932'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:09.008230 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-77e03328-6b55-4316-bb24-6142647f9ceb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0e682af1a7f493c992509142b673932'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:09.051469 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14778f3-6952-45b5-9c61-24629480ce7d None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:09.052429 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14778f3-6952-45b5-9c61-24629480ce7d None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:09.053090 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14778f3-6952-45b5-9c61-24629480ce7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:09.063922 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-77e03328-6b55-4316-bb24-6142647f9ceb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0e682af1a7f493c992509142b673932'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:09.089449 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-e14778f3-6952-45b5-9c61-24629480ce7d None None] MFA Rules not processed for user `19d82c04f9994654adfb10f4bd0573c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:01:09.090868 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e14778f3-6952-45b5-9c61-24629480ce7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:09.100449 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1294/2596] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 15:01:08 2022] DELETE /identity/v3/users/a0e682af1a7f493c992509142b673932 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:01:09.105864 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1303/2597] 10.222.0.42 () {58 vars in 967 bytes} [Wed Jun 15 15:01:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:01:09.119724 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e03328-6b55-4316-bb24-6142647f9ceb admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:09.136034 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-77e03328-6b55-4316-bb24-6142647f9ceb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:09.333353 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e03328-6b55-4316-bb24-6142647f9ceb admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:09.340506 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9282c8c2-c87b-41d0-be0d-13d2d7e1de86 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04dc5832333498e96a7e02bbfb9484c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:09.341327 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9282c8c2-c87b-41d0-be0d-13d2d7e1de86 service swift] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:09.341673 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9282c8c2-c87b-41d0-be0d-13d2d7e1de86 service swift] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:09.342031 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9282c8c2-c87b-41d0-be0d-13d2d7e1de86 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:09.507669 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9282c8c2-c87b-41d0-be0d-13d2d7e1de86 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:09.507669 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9282c8c2-c87b-41d0-be0d-13d2d7e1de86 service swift] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:09.520432 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1295/2598] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Jun 15 15:01:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 15:01:09.628627 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14778f3-6952-45b5-9c61-24629480ce7d None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:09.630580 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e14778f3-6952-45b5-9c61-24629480ce7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:09.676568 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14778f3-6952-45b5-9c61-24629480ce7d None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:09.769486 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:01:09.778986 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bec707-21c8-402d-bc65-cd0b9788108c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:09.785622 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bec707-21c8-402d-bc65-cd0b9788108c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:09.785622 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bec707-21c8-402d-bc65-cd0b9788108c admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:09.785622 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bec707-21c8-402d-bc65-cd0b9788108c admin admin] PATH_INFO: `/v3/projects/f3340f5c4bac49cfaaaa0d80b1bfbb4a` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:09.785622 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bec707-21c8-402d-bc65-cd0b9788108c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3340f5c4bac49cfaaaa0d80b1bfbb4a)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:09.785622 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bec707-21c8-402d-bc65-cd0b9788108c admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:09.934512 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-d4bec707-21c8-402d-bc65-cd0b9788108c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3340f5c4bac49cfaaaa0d80b1bfbb4a', 'request_id': 'req-d4bec707-21c8-402d-bc65-cd0b9788108c'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:09.944649 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-d4bec707-21c8-402d-bc65-cd0b9788108c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3340f5c4bac49cfaaaa0d80b1bfbb4a', 'request_id': 'req-d4bec707-21c8-402d-bc65-cd0b9788108c'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:10.046907 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-d4bec707-21c8-402d-bc65-cd0b9788108c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3340f5c4bac49cfaaaa0d80b1bfbb4a', 'request_id': 'req-d4bec707-21c8-402d-bc65-cd0b9788108c'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:10.062079 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-d4bec707-21c8-402d-bc65-cd0b9788108c admin admin] The token cache is being invalidate because project f3340f5c4bac49cfaaaa0d80b1bfbb4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:01:10.063147 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-d4bec707-21c8-402d-bc65-cd0b9788108c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:10.064439 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-d4bec707-21c8-402d-bc65-cd0b9788108c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:10.066367 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1304/2599] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 15:01:09 2022] DELETE /identity/v3/projects/f3340f5c4bac49cfaaaa0d80b1bfbb4a => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:01:10.070572 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9282c8c2-c87b-41d0-be0d-13d2d7e1de86 service swift] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:10.083834 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9282c8c2-c87b-41d0-be0d-13d2d7e1de86 service swift] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:10.098407 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:01:10.160463 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e30e7-1874-44eb-9f3d-52d074c740a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:10.161565 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e30e7-1874-44eb-9f3d-52d074c740a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:10.161817 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e30e7-1874-44eb-9f3d-52d074c740a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:10.162076 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e30e7-1874-44eb-9f3d-52d074c740a3 admin admin] PATH_INFO: `/v3/users/9a3d6ad33aeb440b821f6efc69a39671` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:10.163324 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2e30e7-1874-44eb-9f3d-52d074c740a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a3d6ad33aeb440b821f6efc69a39671)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:10.165412 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2e30e7-1874-44eb-9f3d-52d074c740a3 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:10.439566 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-df2e30e7-1874-44eb-9f3d-52d074c740a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a3d6ad33aeb440b821f6efc69a39671'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:10.440782 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-df2e30e7-1874-44eb-9f3d-52d074c740a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a3d6ad33aeb440b821f6efc69a39671'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:10.449897 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-df2e30e7-1874-44eb-9f3d-52d074c740a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a3d6ad33aeb440b821f6efc69a39671'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:10.450727 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-df2e30e7-1874-44eb-9f3d-52d074c740a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a3d6ad33aeb440b821f6efc69a39671'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:10.539332 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-df2e30e7-1874-44eb-9f3d-52d074c740a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a3d6ad33aeb440b821f6efc69a39671'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:10.571992 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1296/2600] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 15:01:10 2022] DELETE /identity/v3/users/9a3d6ad33aeb440b821f6efc69a39671 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:01:11.323207 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bec707-21c8-402d-bc65-cd0b9788108c admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:11.324746 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-d4bec707-21c8-402d-bc65-cd0b9788108c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:11.338929 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bec707-21c8-402d-bc65-cd0b9788108c admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:11.405738 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedf609a-1da0-4867-8f35-bcf9664b91d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:11.405738 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedf609a-1da0-4867-8f35-bcf9664b91d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:11.406062 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedf609a-1da0-4867-8f35-bcf9664b91d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:11.406359 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedf609a-1da0-4867-8f35-bcf9664b91d1 admin admin] PATH_INFO: `/v3/projects/646ee05411a1467988f0ad556741c05e` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:11.407565 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedf609a-1da0-4867-8f35-bcf9664b91d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=646ee05411a1467988f0ad556741c05e)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:11.409680 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedf609a-1da0-4867-8f35-bcf9664b91d1 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:11.526615 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-eedf609a-1da0-4867-8f35-bcf9664b91d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '646ee05411a1467988f0ad556741c05e', 'request_id': 'req-eedf609a-1da0-4867-8f35-bcf9664b91d1'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:11.536587 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-eedf609a-1da0-4867-8f35-bcf9664b91d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '646ee05411a1467988f0ad556741c05e', 'request_id': 'req-eedf609a-1da0-4867-8f35-bcf9664b91d1'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:11.624348 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-eedf609a-1da0-4867-8f35-bcf9664b91d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '646ee05411a1467988f0ad556741c05e', 'request_id': 'req-eedf609a-1da0-4867-8f35-bcf9664b91d1'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:11.642695 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-eedf609a-1da0-4867-8f35-bcf9664b91d1 admin admin] The token cache is being invalidate because project 646ee05411a1467988f0ad556741c05e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:01:11.643250 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-eedf609a-1da0-4867-8f35-bcf9664b91d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:11.644030 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-eedf609a-1da0-4867-8f35-bcf9664b91d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:11.645913 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1305/2601] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 15:01:11 2022] DELETE /identity/v3/projects/646ee05411a1467988f0ad556741c05e => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:01:13.404983 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e30e7-1874-44eb-9f3d-52d074c740a3 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:13.408969 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-df2e30e7-1874-44eb-9f3d-52d074c740a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:13.430394 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e30e7-1874-44eb-9f3d-52d074c740a3 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:13.509427 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8ed22b-07d3-42fa-b24d-200d38c28073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:13.510296 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ed22b-07d3-42fa-b24d-200d38c28073 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:13.510700 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ed22b-07d3-42fa-b24d-200d38c28073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:13.510859 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ed22b-07d3-42fa-b24d-200d38c28073 admin admin] PATH_INFO: `/v3/users/33f42c17c1374253b7716ffe0ffbaa6d` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:13.512448 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8ed22b-07d3-42fa-b24d-200d38c28073 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33f42c17c1374253b7716ffe0ffbaa6d)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:13.514702 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8ed22b-07d3-42fa-b24d-200d38c28073 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:13.768652 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:01:13.789193 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-ad8ed22b-07d3-42fa-b24d-200d38c28073 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33f42c17c1374253b7716ffe0ffbaa6d'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:13.789828 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-ad8ed22b-07d3-42fa-b24d-200d38c28073 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33f42c17c1374253b7716ffe0ffbaa6d'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:13.800236 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-ad8ed22b-07d3-42fa-b24d-200d38c28073 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33f42c17c1374253b7716ffe0ffbaa6d'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:13.801092 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-ad8ed22b-07d3-42fa-b24d-200d38c28073 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33f42c17c1374253b7716ffe0ffbaa6d'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:13.922004 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-ad8ed22b-07d3-42fa-b24d-200d38c28073 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33f42c17c1374253b7716ffe0ffbaa6d'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:13.947717 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1297/2602] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 15:01:13 2022] DELETE /identity/v3/users/33f42c17c1374253b7716ffe0ffbaa6d => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:01:14.221409 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedf609a-1da0-4867-8f35-bcf9664b91d1 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:14.223485 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-eedf609a-1da0-4867-8f35-bcf9664b91d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:14.245922 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedf609a-1da0-4867-8f35-bcf9664b91d1 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:14.264699 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:01:14.324566 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcca47cf-2579-4165-ba89-a05546244172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:14.325453 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcca47cf-2579-4165-ba89-a05546244172 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:14.325735 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcca47cf-2579-4165-ba89-a05546244172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:14.326012 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcca47cf-2579-4165-ba89-a05546244172 admin admin] PATH_INFO: `/v3/projects/c0a0e27741434fd282a638aefa713c91` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:14.327265 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcca47cf-2579-4165-ba89-a05546244172 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0a0e27741434fd282a638aefa713c91)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:14.330274 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcca47cf-2579-4165-ba89-a05546244172 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:14.437102 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-fcca47cf-2579-4165-ba89-a05546244172 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0a0e27741434fd282a638aefa713c91', 'request_id': 'req-fcca47cf-2579-4165-ba89-a05546244172'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:14.443958 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-fcca47cf-2579-4165-ba89-a05546244172 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0a0e27741434fd282a638aefa713c91', 'request_id': 'req-fcca47cf-2579-4165-ba89-a05546244172'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:14.601593 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8ed22b-07d3-42fa-b24d-200d38c28073 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:14.603259 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-ad8ed22b-07d3-42fa-b24d-200d38c28073 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:14.668435 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8ed22b-07d3-42fa-b24d-200d38c28073 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:14.724174 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd4f184-4a94-4f55-979c-eb3ecaa644e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:14.724749 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd4f184-4a94-4f55-979c-eb3ecaa644e0 service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:14.725027 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd4f184-4a94-4f55-979c-eb3ecaa644e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:14.725303 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd4f184-4a94-4f55-979c-eb3ecaa644e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:14.727115 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-9fd4f184-4a94-4f55-979c-eb3ecaa644e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:14.733317 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-fcca47cf-2579-4165-ba89-a05546244172 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0a0e27741434fd282a638aefa713c91', 'request_id': 'req-fcca47cf-2579-4165-ba89-a05546244172'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:14.746940 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-fcca47cf-2579-4165-ba89-a05546244172 admin admin] The token cache is being invalidate because project c0a0e27741434fd282a638aefa713c91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:01:14.747218 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-fcca47cf-2579-4165-ba89-a05546244172 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:14.747868 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-fcca47cf-2579-4165-ba89-a05546244172 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:14.749493 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1306/2603] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 15:01:14 2022] DELETE /identity/v3/projects/c0a0e27741434fd282a638aefa713c91 => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:01:14.754904 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bed394-41bb-41bc-b30c-1d7588059961 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:14.755337 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bed394-41bb-41bc-b30c-1d7588059961 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:14.755706 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bed394-41bb-41bc-b30c-1d7588059961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:14.775058 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-64bed394-41bb-41bc-b30c-1d7588059961 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:01:14.776319 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-64bed394-41bb-41bc-b30c-1d7588059961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:14.823594 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd4f184-4a94-4f55-979c-eb3ecaa644e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:14.825261 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd4f184-4a94-4f55-979c-eb3ecaa644e0 service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:14.836829 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:01:14.841874 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1307/2604] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:01:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:01:14.846877 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1298/2605] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:01:14 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 15:01:14.850427 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c002ae66-56e9-4801-a0b2-a0a47b10f235 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:14.850676 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c002ae66-56e9-4801-a0b2-a0a47b10f235 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:14.850937 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c002ae66-56e9-4801-a0b2-a0a47b10f235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:14.875236 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-c002ae66-56e9-4801-a0b2-a0a47b10f235 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:01:14.876536 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c002ae66-56e9-4801-a0b2-a0a47b10f235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:14.886378 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1308/2606] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:01:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:01:14.890477 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd4f184-4a94-4f55-979c-eb3ecaa644e0 service cinder] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:14.892653 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-9fd4f184-4a94-4f55-979c-eb3ecaa644e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:14.906712 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd4f184-4a94-4f55-979c-eb3ecaa644e0 service cinder] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:14.913535 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b574f49-8b87-4c82-a3d9-1215e258b3ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:14.917067 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b574f49-8b87-4c82-a3d9-1215e258b3ca admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:14.917067 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b574f49-8b87-4c82-a3d9-1215e258b3ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:14.917067 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b574f49-8b87-4c82-a3d9-1215e258b3ca admin admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:14.917067 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b574f49-8b87-4c82-a3d9-1215e258b3ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:01:14.917067 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b574f49-8b87-4c82-a3d9-1215e258b3ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:14.917606 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b574f49-8b87-4c82-a3d9-1215e258b3ca admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:14.931258 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1299/2607] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jun 15 15:01:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:01:14.935855 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c002ae66-56e9-4801-a0b2-a0a47b10f235 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:14.946225 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c002ae66-56e9-4801-a0b2-a0a47b10f235 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:14.951360 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41789b8-5fe0-4d90-ae5a-7142b85071b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:14.952149 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41789b8-5fe0-4d90-ae5a-7142b85071b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:14.952514 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41789b8-5fe0-4d90-ae5a-7142b85071b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:14.952900 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41789b8-5fe0-4d90-ae5a-7142b85071b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:14.953673 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41789b8-5fe0-4d90-ae5a-7142b85071b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:14.956465 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41789b8-5fe0-4d90-ae5a-7142b85071b6 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:15.060002 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1309/2608] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 15:01:14 2022] POST /identity/v3/projects => generated 377 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:01:15.061015 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b574f49-8b87-4c82-a3d9-1215e258b3ca admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:15.073033 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b574f49-8b87-4c82-a3d9-1215e258b3ca admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:15.139441 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabf7761-293f-410d-b388-e1b04012f03f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:15.140175 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf7761-293f-410d-b388-e1b04012f03f admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:15.140470 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf7761-293f-410d-b388-e1b04012f03f admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:15.140914 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf7761-293f-410d-b388-e1b04012f03f admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:15.142233 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabf7761-293f-410d-b388-e1b04012f03f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:15.144398 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabf7761-293f-410d-b388-e1b04012f03f admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:15.263172 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1300/2609] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 15:01:15 2022] POST /identity/v3/users => generated 289 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:01:15.269356 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41789b8-5fe0-4d90-ae5a-7142b85071b6 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:15.291286 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41789b8-5fe0-4d90-ae5a-7142b85071b6 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:15.298917 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5974ed4-4467-481e-b31d-60bf9caea99a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:15.300018 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5974ed4-4467-481e-b31d-60bf9caea99a admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:15.300754 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5974ed4-4467-481e-b31d-60bf9caea99a admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:15.301431 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5974ed4-4467-481e-b31d-60bf9caea99a admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:15.303809 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5974ed4-4467-481e-b31d-60bf9caea99a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:15.307110 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5974ed4-4467-481e-b31d-60bf9caea99a admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:15.325253 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1310/2610] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 15:01:15 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:01:15.326737 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabf7761-293f-410d-b388-e1b04012f03f admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:15.339774 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabf7761-293f-410d-b388-e1b04012f03f admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:15.345886 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1b5d02-bccd-4c22-9d9e-39ada7e59cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:15.346988 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b5d02-bccd-4c22-9d9e-39ada7e59cf1 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:15.346988 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b5d02-bccd-4c22-9d9e-39ada7e59cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:15.347394 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b5d02-bccd-4c22-9d9e-39ada7e59cf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:15.348926 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1b5d02-bccd-4c22-9d9e-39ada7e59cf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:15.349896 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1b5d02-bccd-4c22-9d9e-39ada7e59cf1 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:15.361245 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1301/2611] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 15:01:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:01:15.365404 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5974ed4-4467-481e-b31d-60bf9caea99a admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:15.388349 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5974ed4-4467-481e-b31d-60bf9caea99a admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:15.394259 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e559f9fd-4da5-4d7f-8e71-5ea2d7beb2ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:15.395468 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e559f9fd-4da5-4d7f-8e71-5ea2d7beb2ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:15.395753 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e559f9fd-4da5-4d7f-8e71-5ea2d7beb2ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:15.396033 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e559f9fd-4da5-4d7f-8e71-5ea2d7beb2ba admin admin] PATH_INFO: `/v3/projects/79f361ea99aa45c185d50d83c84e5dc1/users/e209ae2f7fa7412b95c7c43c74c14426/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:15.399613 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:01:15.414019 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e559f9fd-4da5-4d7f-8e71-5ea2d7beb2ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=79f361ea99aa45c185d50d83c84e5dc1, user_id=e209ae2f7fa7412b95c7c43c74c14426, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:15.417897 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e559f9fd-4da5-4d7f-8e71-5ea2d7beb2ba admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:15.477821 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1311/2612] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 15:01:15 2022] PUT /identity/v3/projects/79f361ea99aa45c185d50d83c84e5dc1/users/e209ae2f7fa7412b95c7c43c74c14426/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:01:15.483140 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7f46b9-9101-4335-bc31-9b570dc619b3 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:15.483534 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7f46b9-9101-4335-bc31-9b570dc619b3 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:15.483910 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7f46b9-9101-4335-bc31-9b570dc619b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:15.509385 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-5b7f46b9-9101-4335-bc31-9b570dc619b3 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:01:15.510641 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-5b7f46b9-9101-4335-bc31-9b570dc619b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:15.587638 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1302/2613] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:01:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:01:15.597499 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e559f9fd-4da5-4d7f-8e71-5ea2d7beb2ba admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:15.599263 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e559f9fd-4da5-4d7f-8e71-5ea2d7beb2ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:15.666776 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e559f9fd-4da5-4d7f-8e71-5ea2d7beb2ba admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:15.671315 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6c08ff-eb37-4d83-b68a-623b832ebdfc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:15.672067 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c08ff-eb37-4d83-b68a-623b832ebdfc service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:15.672488 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c08ff-eb37-4d83-b68a-623b832ebdfc service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:15.672852 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c08ff-eb37-4d83-b68a-623b832ebdfc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:15.683655 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6c08ff-eb37-4d83-b68a-623b832ebdfc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:15.685290 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6c08ff-eb37-4d83-b68a-623b832ebdfc service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:15.696132 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1312/2614] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:01:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:01:22.130187 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735dff14-d6ed-4711-be70-a9e67c8d21b7 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:22.130609 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735dff14-d6ed-4711-be70-a9e67c8d21b7 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:22.130695 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735dff14-d6ed-4711-be70-a9e67c8d21b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:22.163585 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-735dff14-d6ed-4711-be70-a9e67c8d21b7 None None] MFA Rules not processed for user `e209ae2f7fa7412b95c7c43c74c14426`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:01:22.164790 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-735dff14-d6ed-4711-be70-a9e67c8d21b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:22.241694 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:01:22.261770 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1303/2615] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:01:22 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:01:22.266609 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6c08ff-eb37-4d83-b68a-623b832ebdfc service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:22.277634 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6c08ff-eb37-4d83-b68a-623b832ebdfc service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:22.283109 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68273c98-0159-46ee-9e3e-18269b674864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:22.283766 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68273c98-0159-46ee-9e3e-18269b674864 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:22.284045 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68273c98-0159-46ee-9e3e-18269b674864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:22.284348 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68273c98-0159-46ee-9e3e-18269b674864 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:22.285047 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68273c98-0159-46ee-9e3e-18269b674864 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:22.286926 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68273c98-0159-46ee-9e3e-18269b674864 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:22.408975 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1313/2616] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 15:01:22 2022] POST /identity/v3/projects => generated 379 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:01:22.413389 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735dff14-d6ed-4711-be70-a9e67c8d21b7 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:22.425224 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735dff14-d6ed-4711-be70-a9e67c8d21b7 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:22.483740 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8803c0-9d2b-4af7-aa6c-5ddb0a9165ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:22.484512 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8803c0-9d2b-4af7-aa6c-5ddb0a9165ca admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:22.484880 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8803c0-9d2b-4af7-aa6c-5ddb0a9165ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:22.485179 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8803c0-9d2b-4af7-aa6c-5ddb0a9165ca admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:22.485895 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8803c0-9d2b-4af7-aa6c-5ddb0a9165ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:22.487861 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8803c0-9d2b-4af7-aa6c-5ddb0a9165ca admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:22.591557 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1304/2617] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 15:01:22 2022] POST /identity/v3/users => generated 296 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:01:22.596283 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68273c98-0159-46ee-9e3e-18269b674864 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:22.607488 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68273c98-0159-46ee-9e3e-18269b674864 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:22.612789 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1193f61-c9a8-44bf-a303-582022d28cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:22.613473 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1193f61-c9a8-44bf-a303-582022d28cc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:22.613862 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1193f61-c9a8-44bf-a303-582022d28cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:22.614042 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1193f61-c9a8-44bf-a303-582022d28cc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:22.614962 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1193f61-c9a8-44bf-a303-582022d28cc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:22.616624 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1193f61-c9a8-44bf-a303-582022d28cc4 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:22.628905 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1314/2618] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 15:01:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 15:01:22.633102 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8803c0-9d2b-4af7-aa6c-5ddb0a9165ca admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:22.645407 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8803c0-9d2b-4af7-aa6c-5ddb0a9165ca admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:22.650860 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3d34d7-e749-4822-b0fa-16363647c457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:22.651833 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d34d7-e749-4822-b0fa-16363647c457 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:22.652327 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d34d7-e749-4822-b0fa-16363647c457 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:22.652677 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d34d7-e749-4822-b0fa-16363647c457 admin admin] PATH_INFO: `/v3/projects/0e3cf089909f446981d63c658e07666b/users/6bc416dafea64c64bf5d472e908daacb/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:22.670079 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3d34d7-e749-4822-b0fa-16363647c457 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e3cf089909f446981d63c658e07666b, user_id=6bc416dafea64c64bf5d472e908daacb, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:22.674163 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3d34d7-e749-4822-b0fa-16363647c457 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:22.755800 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1305/2619] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 15:01:22 2022] PUT /identity/v3/projects/0e3cf089909f446981d63c658e07666b/users/6bc416dafea64c64bf5d472e908daacb/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:01:22.759814 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1193f61-c9a8-44bf-a303-582022d28cc4 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:22.772499 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1193f61-c9a8-44bf-a303-582022d28cc4 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:22.839398 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1fa4a1-e420-4c02-88e1-b8a7c79762af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:22.840070 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1fa4a1-e420-4c02-88e1-b8a7c79762af admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:22.840418 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1fa4a1-e420-4c02-88e1-b8a7c79762af admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:22.840695 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1fa4a1-e420-4c02-88e1-b8a7c79762af admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:22.841418 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1fa4a1-e420-4c02-88e1-b8a7c79762af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:22.842993 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1fa4a1-e420-4c02-88e1-b8a7c79762af admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:22.853811 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1315/2620] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 15:01:22 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:01:22.858050 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3d34d7-e749-4822-b0fa-16363647c457 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:22.868724 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3d34d7-e749-4822-b0fa-16363647c457 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:22.874663 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8824bb30-203a-4fbc-a69a-5f1d66930edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:22.877066 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8824bb30-203a-4fbc-a69a-5f1d66930edf admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:22.877366 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8824bb30-203a-4fbc-a69a-5f1d66930edf admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:22.877637 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8824bb30-203a-4fbc-a69a-5f1d66930edf admin admin] PATH_INFO: `/v3/projects/0e3cf089909f446981d63c658e07666b/users/6bc416dafea64c64bf5d472e908daacb/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:22.879771 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8824bb30-203a-4fbc-a69a-5f1d66930edf admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e3cf089909f446981d63c658e07666b, user_id=6bc416dafea64c64bf5d472e908daacb, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:22.882837 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8824bb30-203a-4fbc-a69a-5f1d66930edf admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:22.906247 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1306/2621] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 15:01:22 2022] PUT /identity/v3/projects/0e3cf089909f446981d63c658e07666b/users/6bc416dafea64c64bf5d472e908daacb/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:01:27.050068 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1fa4a1-e420-4c02-88e1-b8a7c79762af admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:27.054980 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-dc1fa4a1-e420-4c02-88e1-b8a7c79762af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:27.141577 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1fa4a1-e420-4c02-88e1-b8a7c79762af admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:27.146527 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:01:27.205554 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4497c69-d0d2-4986-bcce-3e33638ab23b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f18c57f27784694935bfb0bcea48a78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:27.206553 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4497c69-d0d2-4986-bcce-3e33638ab23b service glance] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:27.206985 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4497c69-d0d2-4986-bcce-3e33638ab23b service glance] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:27.207434 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4497c69-d0d2-4986-bcce-3e33638ab23b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:27.209782 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-f4497c69-d0d2-4986-bcce-3e33638ab23b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:27.309785 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4497c69-d0d2-4986-bcce-3e33638ab23b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:27.311849 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4497c69-d0d2-4986-bcce-3e33638ab23b service glance] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:27.339638 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1316/2622] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:01:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:01:27.633042 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8824bb30-203a-4fbc-a69a-5f1d66930edf admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:27.635674 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-8824bb30-203a-4fbc-a69a-5f1d66930edf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:27.725536 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8824bb30-203a-4fbc-a69a-5f1d66930edf admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:27.731233 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05241305-320f-459e-b367-87e3d3a29d5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:27.732696 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05241305-320f-459e-b367-87e3d3a29d5e service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:27.733191 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05241305-320f-459e-b367-87e3d3a29d5e service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:27.733531 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05241305-320f-459e-b367-87e3d3a29d5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:27.736849 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-05241305-320f-459e-b367-87e3d3a29d5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:27.755479 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05241305-320f-459e-b367-87e3d3a29d5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:27.762335 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05241305-320f-459e-b367-87e3d3a29d5e service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:27.767828 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1307/2623] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:01:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:01:27.887719 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4497c69-d0d2-4986-bcce-3e33638ab23b service glance] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:27.898635 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4497c69-d0d2-4986-bcce-3e33638ab23b service glance] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:27.902890 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0d2d86-6873-4169-bd92-d88554ed73f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:27.903544 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0d2d86-6873-4169-bd92-d88554ed73f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:27.903836 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0d2d86-6873-4169-bd92-d88554ed73f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:27.904143 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0d2d86-6873-4169-bd92-d88554ed73f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:27.906071 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-bd0d2d86-6873-4169-bd92-d88554ed73f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:27.923478 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0d2d86-6873-4169-bd92-d88554ed73f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:27.925179 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0d2d86-6873-4169-bd92-d88554ed73f5 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:27.939669 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1317/2624] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:01:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:01:28.035210 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05241305-320f-459e-b367-87e3d3a29d5e service neutron] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:28.036820 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-05241305-320f-459e-b367-87e3d3a29d5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:28.129834 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:01:28.182329 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05241305-320f-459e-b367-87e3d3a29d5e service neutron] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:28.190010 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967a873d-20bd-46ea-a460-ef77e04a0fe7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bb6671c19464424a6e9cd4519ac93e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:28.191018 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a873d-20bd-46ea-a460-ef77e04a0fe7 service placement] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:28.191453 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a873d-20bd-46ea-a460-ef77e04a0fe7 service placement] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:28.191903 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a873d-20bd-46ea-a460-ef77e04a0fe7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:28.194230 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-967a873d-20bd-46ea-a460-ef77e04a0fe7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:28.209824 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967a873d-20bd-46ea-a460-ef77e04a0fe7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:28.211740 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967a873d-20bd-46ea-a460-ef77e04a0fe7 service placement] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:28.223799 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1308/2625] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:01:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:01:29.991198 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a5d1f-f866-473b-83d9-d933a0cc7c1d None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:29.991709 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a5d1f-f866-473b-83d9-d933a0cc7c1d None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:29.991820 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a5d1f-f866-473b-83d9-d933a0cc7c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:30.023810 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-981a5d1f-f866-473b-83d9-d933a0cc7c1d None None] MFA Rules not processed for user `6bc416dafea64c64bf5d472e908daacb`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:01:30.024557 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-981a5d1f-f866-473b-83d9-d933a0cc7c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:30.153464 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1318/2626] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:01:29 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:01:30.169148 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967a873d-20bd-46ea-a460-ef77e04a0fe7 service placement] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:30.170677 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-967a873d-20bd-46ea-a460-ef77e04a0fe7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:30.187034 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967a873d-20bd-46ea-a460-ef77e04a0fe7 service placement] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:30.194567 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22772db5-8fb3-43ca-972e-e4950fd52c54 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0927c05c340c48e48b18754888aa9c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:30.195322 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22772db5-8fb3-43ca-972e-e4950fd52c54 service nova] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:30.195675 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22772db5-8fb3-43ca-972e-e4950fd52c54 service nova] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:30.196037 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22772db5-8fb3-43ca-972e-e4950fd52c54 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:30.288710 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22772db5-8fb3-43ca-972e-e4950fd52c54 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:30.290512 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22772db5-8fb3-43ca-972e-e4950fd52c54 service nova] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:30.307003 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1309/2627] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:01:30 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:01:31.480672 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580edaa0-f9df-471d-8a9e-1b1b21e02aa2 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:31.483680 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580edaa0-f9df-471d-8a9e-1b1b21e02aa2 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:31.483680 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580edaa0-f9df-471d-8a9e-1b1b21e02aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:31.500959 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-580edaa0-f9df-471d-8a9e-1b1b21e02aa2 None None] MFA Rules not processed for user `19d82c04f9994654adfb10f4bd0573c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:01:31.502214 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-580edaa0-f9df-471d-8a9e-1b1b21e02aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:31.516762 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:01:31.575542 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1319/2628] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Jun 15 15:01:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:01:31.597952 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22772db5-8fb3-43ca-972e-e4950fd52c54 service nova] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:31.605080 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-22772db5-8fb3-43ca-972e-e4950fd52c54 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:31.749506 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22772db5-8fb3-43ca-972e-e4950fd52c54 service nova] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:31.768999 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9771e024-26fb-49a8-903c-169152fc5f19 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04dc5832333498e96a7e02bbfb9484c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:31.769773 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9771e024-26fb-49a8-903c-169152fc5f19 service swift] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:31.770135 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9771e024-26fb-49a8-903c-169152fc5f19 service swift] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:31.770718 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9771e024-26fb-49a8-903c-169152fc5f19 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:31.783288 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9771e024-26fb-49a8-903c-169152fc5f19 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:31.785543 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9771e024-26fb-49a8-903c-169152fc5f19 service swift] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:31.808473 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1310/2629] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Jun 15 15:01:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 15:01:38.009843 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580edaa0-f9df-471d-8a9e-1b1b21e02aa2 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:38.021473 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580edaa0-f9df-471d-8a9e-1b1b21e02aa2 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:38.028246 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e9e112-f583-487c-9623-ee0b13fb6542 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:38.029106 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e9e112-f583-487c-9623-ee0b13fb6542 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:38.029499 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e9e112-f583-487c-9623-ee0b13fb6542 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:38.030050 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e9e112-f583-487c-9623-ee0b13fb6542 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:38.032966 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-02e9e112-f583-487c-9623-ee0b13fb6542 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:38.125389 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e9e112-f583-487c-9623-ee0b13fb6542 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:38.127010 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e9e112-f583-487c-9623-ee0b13fb6542 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:38.142975 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1320/2630] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:01:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:01:41.793886 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=71 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:01:42.640602 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9771e024-26fb-49a8-903c-169152fc5f19 service swift] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:42.642872 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-9771e024-26fb-49a8-903c-169152fc5f19 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:42.661926 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9771e024-26fb-49a8-903c-169152fc5f19 service swift] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:42.666774 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26560b4-9008-44ee-8ec1-83722c5c2283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:42.667673 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26560b4-9008-44ee-8ec1-83722c5c2283 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:42.667952 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26560b4-9008-44ee-8ec1-83722c5c2283 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:42.668260 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26560b4-9008-44ee-8ec1-83722c5c2283 admin admin] PATH_INFO: `/v3/users/0650eb65117a4e27908c4355bfc46c3b` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:42.669478 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26560b4-9008-44ee-8ec1-83722c5c2283 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0650eb65117a4e27908c4355bfc46c3b)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:42.671589 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26560b4-9008-44ee-8ec1-83722c5c2283 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:42.854143 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-c26560b4-9008-44ee-8ec1-83722c5c2283 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0650eb65117a4e27908c4355bfc46c3b'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:42.854848 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-c26560b4-9008-44ee-8ec1-83722c5c2283 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0650eb65117a4e27908c4355bfc46c3b'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:42.863513 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-c26560b4-9008-44ee-8ec1-83722c5c2283 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0650eb65117a4e27908c4355bfc46c3b'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:42.864298 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-c26560b4-9008-44ee-8ec1-83722c5c2283 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0650eb65117a4e27908c4355bfc46c3b'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:42.931082 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-c26560b4-9008-44ee-8ec1-83722c5c2283 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0650eb65117a4e27908c4355bfc46c3b'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:42.950287 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1311/2631] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 15:01:42 2022] DELETE /identity/v3/users/0650eb65117a4e27908c4355bfc46c3b => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:01:43.213973 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e9e112-f583-487c-9623-ee0b13fb6542 service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:43.225249 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e9e112-f583-487c-9623-ee0b13fb6542 service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:43.291848 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61683c23-2374-4bf2-85de-28dfe3f38bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:43.292866 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61683c23-2374-4bf2-85de-28dfe3f38bea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:43.293147 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61683c23-2374-4bf2-85de-28dfe3f38bea admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:43.293481 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61683c23-2374-4bf2-85de-28dfe3f38bea admin admin] PATH_INFO: `/v3/projects/175b90b822a942bd8e04b8ad7d228a57` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:43.294605 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61683c23-2374-4bf2-85de-28dfe3f38bea admin admin] RBAC: Authorizing `identity:delete_project(project_id=175b90b822a942bd8e04b8ad7d228a57)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:43.296638 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61683c23-2374-4bf2-85de-28dfe3f38bea admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:43.397363 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-61683c23-2374-4bf2-85de-28dfe3f38bea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '175b90b822a942bd8e04b8ad7d228a57', 'request_id': 'req-61683c23-2374-4bf2-85de-28dfe3f38bea'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:43.403596 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-61683c23-2374-4bf2-85de-28dfe3f38bea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '175b90b822a942bd8e04b8ad7d228a57', 'request_id': 'req-61683c23-2374-4bf2-85de-28dfe3f38bea'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:43.457562 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-61683c23-2374-4bf2-85de-28dfe3f38bea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '175b90b822a942bd8e04b8ad7d228a57', 'request_id': 'req-61683c23-2374-4bf2-85de-28dfe3f38bea'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:43.459012 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:01:43.469298 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-61683c23-2374-4bf2-85de-28dfe3f38bea admin admin] The token cache is being invalidate because project 175b90b822a942bd8e04b8ad7d228a57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:01:43.469586 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-61683c23-2374-4bf2-85de-28dfe3f38bea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:43.470270 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-61683c23-2374-4bf2-85de-28dfe3f38bea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:43.471921 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1321/2632] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 15:01:43 2022] DELETE /identity/v3/projects/175b90b822a942bd8e04b8ad7d228a57 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:01:43.477076 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dafa770-6e56-420a-83ff-80e8015a6541 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:43.477366 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dafa770-6e56-420a-83ff-80e8015a6541 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:43.477645 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dafa770-6e56-420a-83ff-80e8015a6541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:43.497511 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-0dafa770-6e56-420a-83ff-80e8015a6541 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:01:43.498716 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-0dafa770-6e56-420a-83ff-80e8015a6541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:43.565584 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1312/2633] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:01:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:01:43.573995 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e36d2f-b138-470d-94d7-85ba83937d9f None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:43.574298 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e36d2f-b138-470d-94d7-85ba83937d9f None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:43.574606 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e36d2f-b138-470d-94d7-85ba83937d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:43.594562 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-b0e36d2f-b138-470d-94d7-85ba83937d9f None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:01:43.596041 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-b0e36d2f-b138-470d-94d7-85ba83937d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:43.604835 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1322/2634] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:01:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:01:43.609105 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dafa770-6e56-420a-83ff-80e8015a6541 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:43.619859 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dafa770-6e56-420a-83ff-80e8015a6541 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:43.624860 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81e9948-611f-41aa-b99a-addaba6d98d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:43.626176 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81e9948-611f-41aa-b99a-addaba6d98d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:43.626461 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81e9948-611f-41aa-b99a-addaba6d98d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:43.627070 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81e9948-611f-41aa-b99a-addaba6d98d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:43.627589 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81e9948-611f-41aa-b99a-addaba6d98d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:01:43.627815 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81e9948-611f-41aa-b99a-addaba6d98d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:43.629450 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81e9948-611f-41aa-b99a-addaba6d98d4 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:43.641930 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1313/2635] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jun 15 15:01:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:01:43.646271 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e36d2f-b138-470d-94d7-85ba83937d9f None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:43.656708 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e36d2f-b138-470d-94d7-85ba83937d9f None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:43.661851 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7949af50-1999-49b0-8079-f46f5cdb4105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:43.662512 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7949af50-1999-49b0-8079-f46f5cdb4105 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:43.662791 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7949af50-1999-49b0-8079-f46f5cdb4105 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:43.663065 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7949af50-1999-49b0-8079-f46f5cdb4105 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:43.663803 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7949af50-1999-49b0-8079-f46f5cdb4105 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:43.665764 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7949af50-1999-49b0-8079-f46f5cdb4105 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:43.740812 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1323/2636] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 15:01:43 2022] POST /identity/v3/projects => generated 367 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:01:43.745167 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81e9948-611f-41aa-b99a-addaba6d98d4 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:43.758355 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81e9948-611f-41aa-b99a-addaba6d98d4 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:43.817546 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57321af-551b-419e-88ca-761e7553fc79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:43.818100 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57321af-551b-419e-88ca-761e7553fc79 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:43.818396 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57321af-551b-419e-88ca-761e7553fc79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:43.818682 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57321af-551b-419e-88ca-761e7553fc79 admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:43.819416 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57321af-551b-419e-88ca-761e7553fc79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:43.821361 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57321af-551b-419e-88ca-761e7553fc79 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:43.828123 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:01:43.930697 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1314/2637] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 15:01:43 2022] POST /identity/v3/users => generated 284 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:01:43.937536 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7949af50-1999-49b0-8079-f46f5cdb4105 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:43.950830 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7949af50-1999-49b0-8079-f46f5cdb4105 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:43.956304 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03724b10-2abd-4211-83d2-cd62882ea726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:43.956994 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03724b10-2abd-4211-83d2-cd62882ea726 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:43.957276 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03724b10-2abd-4211-83d2-cd62882ea726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:43.957551 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03724b10-2abd-4211-83d2-cd62882ea726 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:43.958881 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03724b10-2abd-4211-83d2-cd62882ea726 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:43.959903 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03724b10-2abd-4211-83d2-cd62882ea726 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:43.980696 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1324/2638] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 15:01:43 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:01:43.984552 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57321af-551b-419e-88ca-761e7553fc79 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:43.997331 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57321af-551b-419e-88ca-761e7553fc79 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:44.002720 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9f68aa-8d14-4e7f-80b6-65f59c44ab81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:44.003982 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9f68aa-8d14-4e7f-80b6-65f59c44ab81 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:44.003982 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9f68aa-8d14-4e7f-80b6-65f59c44ab81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:44.004735 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9f68aa-8d14-4e7f-80b6-65f59c44ab81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:44.005122 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9f68aa-8d14-4e7f-80b6-65f59c44ab81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:44.007010 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9f68aa-8d14-4e7f-80b6-65f59c44ab81 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:44.023139 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1315/2639] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 15:01:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 15:01:44.026400 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03724b10-2abd-4211-83d2-cd62882ea726 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:44.038868 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03724b10-2abd-4211-83d2-cd62882ea726 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:44.043887 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b847c337-5e0a-4100-b163-a53fdbe84d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:44.044899 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b847c337-5e0a-4100-b163-a53fdbe84d42 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:44.045178 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b847c337-5e0a-4100-b163-a53fdbe84d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:44.045451 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b847c337-5e0a-4100-b163-a53fdbe84d42 admin admin] PATH_INFO: `/v3/projects/25e7a4bfab9844b79ac0430d7807cbb2/users/c85a0d3100854dc2be3cdf69935e53b8/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:44.062943 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b847c337-5e0a-4100-b163-a53fdbe84d42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25e7a4bfab9844b79ac0430d7807cbb2, user_id=c85a0d3100854dc2be3cdf69935e53b8, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:44.069166 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b847c337-5e0a-4100-b163-a53fdbe84d42 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:44.162569 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1325/2640] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 15:01:44 2022] PUT /identity/v3/projects/25e7a4bfab9844b79ac0430d7807cbb2/users/c85a0d3100854dc2be3cdf69935e53b8/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:01:44.173312 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f23ed-fbb8-4897-a21b-ca2978f03af6 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:44.173593 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f23ed-fbb8-4897-a21b-ca2978f03af6 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:44.173867 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f23ed-fbb8-4897-a21b-ca2978f03af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:44.201076 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-b00f23ed-fbb8-4897-a21b-ca2978f03af6 None None] MFA Rules not processed for user `c85a0d3100854dc2be3cdf69935e53b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:01:44.202286 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-b00f23ed-fbb8-4897-a21b-ca2978f03af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:44.269449 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1316/2641] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:01:44 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:01:44.280576 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b847c337-5e0a-4100-b163-a53fdbe84d42 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:44.282873 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-b847c337-5e0a-4100-b163-a53fdbe84d42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:44.345433 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b847c337-5e0a-4100-b163-a53fdbe84d42 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:44.349639 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53717e5-b49a-47fc-a739-ba73648dd3b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:44.350338 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53717e5-b49a-47fc-a739-ba73648dd3b3 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:44.350623 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53717e5-b49a-47fc-a739-ba73648dd3b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:44.350906 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53717e5-b49a-47fc-a739-ba73648dd3b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:44.360273 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53717e5-b49a-47fc-a739-ba73648dd3b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:44.362114 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53717e5-b49a-47fc-a739-ba73648dd3b3 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:44.372540 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1326/2642] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:01:44 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:01:52.388783 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00f23ed-fbb8-4897-a21b-ca2978f03af6 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:52.392479 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-b00f23ed-fbb8-4897-a21b-ca2978f03af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:52.468557 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00f23ed-fbb8-4897-a21b-ca2978f03af6 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:52.474498 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c930ae-b238-49af-977d-e4f91952bc54 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0927c05c340c48e48b18754888aa9c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:52.480366 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c930ae-b238-49af-977d-e4f91952bc54 service nova] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:52.480366 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c930ae-b238-49af-977d-e4f91952bc54 service nova] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:52.480366 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c930ae-b238-49af-977d-e4f91952bc54 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:52.480366 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-d4c930ae-b238-49af-977d-e4f91952bc54 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:52.491330 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c930ae-b238-49af-977d-e4f91952bc54 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:52.492938 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c930ae-b238-49af-977d-e4f91952bc54 service nova] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:52.505391 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1317/2643] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:01:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:01:54.367931 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:DELETE=2,keystone:SELECT=56,keystone:INSERT=2 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:01:55.049250 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d46df9-258f-4876-a699-5ae004478db9 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:55.049534 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d46df9-258f-4876-a699-5ae004478db9 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:55.049813 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d46df9-258f-4876-a699-5ae004478db9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:55.081840 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-51d46df9-258f-4876-a699-5ae004478db9 None None] MFA Rules not processed for user `19d82c04f9994654adfb10f4bd0573c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:01:55.083707 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-51d46df9-258f-4876-a699-5ae004478db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:55.140104 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1327/2644] 10.222.0.42 () {58 vars in 967 bytes} [Wed Jun 15 15:01:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:01:55.165355 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c930ae-b238-49af-977d-e4f91952bc54 service nova] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:55.166917 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-d4c930ae-b238-49af-977d-e4f91952bc54 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:55.227103 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:01:55.258228 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c930ae-b238-49af-977d-e4f91952bc54 service nova] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:55.264119 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd0f938-0def-4557-a530-e68f2ae54322 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04dc5832333498e96a7e02bbfb9484c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:55.264791 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd0f938-0def-4557-a530-e68f2ae54322 service swift] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:55.265136 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd0f938-0def-4557-a530-e68f2ae54322 service swift] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:55.265414 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd0f938-0def-4557-a530-e68f2ae54322 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:55.274680 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd0f938-0def-4557-a530-e68f2ae54322 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:55.276324 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd0f938-0def-4557-a530-e68f2ae54322 service swift] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:55.284321 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1318/2645] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Jun 15 15:01:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 15 15:01:57.538022 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d46df9-258f-4876-a699-5ae004478db9 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:57.620623 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d46df9-258f-4876-a699-5ae004478db9 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:57.625728 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaea25b-660e-4e18-836f-4539f43169e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:57.626602 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaea25b-660e-4e18-836f-4539f43169e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:57.626879 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaea25b-660e-4e18-836f-4539f43169e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:57.627151 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaea25b-660e-4e18-836f-4539f43169e0 admin admin] PATH_INFO: `/v3/users/c85a0d3100854dc2be3cdf69935e53b8` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:57.628499 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaea25b-660e-4e18-836f-4539f43169e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c85a0d3100854dc2be3cdf69935e53b8)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:57.630662 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaea25b-660e-4e18-836f-4539f43169e0 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:57.831745 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5eaea25b-660e-4e18-836f-4539f43169e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c85a0d3100854dc2be3cdf69935e53b8'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:57.832368 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5eaea25b-660e-4e18-836f-4539f43169e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c85a0d3100854dc2be3cdf69935e53b8'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:57.833492 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5eaea25b-660e-4e18-836f-4539f43169e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c85a0d3100854dc2be3cdf69935e53b8'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:57.909374 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5eaea25b-660e-4e18-836f-4539f43169e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c85a0d3100854dc2be3cdf69935e53b8'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:57.924470 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5eaea25b-660e-4e18-836f-4539f43169e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c85a0d3100854dc2be3cdf69935e53b8'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:57.932876 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:01:57.938157 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1328/2646] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 15:01:57 2022] DELETE /identity/v3/users/c85a0d3100854dc2be3cdf69935e53b8 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:01:57.943807 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d46cd-8b6e-460e-b67e-e069c3ea4024 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:57.944355 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d46cd-8b6e-460e-b67e-e069c3ea4024 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:57.944459 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d46cd-8b6e-460e-b67e-e069c3ea4024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:57.975159 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-c59d46cd-8b6e-460e-b67e-e069c3ea4024 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:01:57.976376 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-c59d46cd-8b6e-460e-b67e-e069c3ea4024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:58.042452 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1319/2647] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:01:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:01:58.052896 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaea25b-660e-4e18-836f-4539f43169e0 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:58.054296 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-5eaea25b-660e-4e18-836f-4539f43169e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:58.143796 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaea25b-660e-4e18-836f-4539f43169e0 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:58.148021 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c465c0a-cc9c-4578-ae0d-a75e57fa5f2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:58.148703 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c465c0a-cc9c-4578-ae0d-a75e57fa5f2c service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:58.148986 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c465c0a-cc9c-4578-ae0d-a75e57fa5f2c service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:58.149261 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c465c0a-cc9c-4578-ae0d-a75e57fa5f2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:58.158647 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c465c0a-cc9c-4578-ae0d-a75e57fa5f2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:58.160082 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c465c0a-cc9c-4578-ae0d-a75e57fa5f2c service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:58.171997 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1329/2648] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:01:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:01:58.659799 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59d46cd-8b6e-460e-b67e-e069c3ea4024 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:58.662047 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-c59d46cd-8b6e-460e-b67e-e069c3ea4024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:58.675992 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59d46cd-8b6e-460e-b67e-e069c3ea4024 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:58.681739 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0911471f-1af9-4796-b6ee-d64889372e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:58.682605 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0911471f-1af9-4796-b6ee-d64889372e00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:58.682888 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0911471f-1af9-4796-b6ee-d64889372e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:58.683197 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0911471f-1af9-4796-b6ee-d64889372e00 admin admin] PATH_INFO: `/v3/projects/25e7a4bfab9844b79ac0430d7807cbb2` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:58.684640 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0911471f-1af9-4796-b6ee-d64889372e00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25e7a4bfab9844b79ac0430d7807cbb2)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:58.686568 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0911471f-1af9-4796-b6ee-d64889372e00 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:58.853742 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-0911471f-1af9-4796-b6ee-d64889372e00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25e7a4bfab9844b79ac0430d7807cbb2', 'request_id': 'req-0911471f-1af9-4796-b6ee-d64889372e00'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:58.949241 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-0911471f-1af9-4796-b6ee-d64889372e00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25e7a4bfab9844b79ac0430d7807cbb2', 'request_id': 'req-0911471f-1af9-4796-b6ee-d64889372e00'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:58.962857 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-0911471f-1af9-4796-b6ee-d64889372e00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25e7a4bfab9844b79ac0430d7807cbb2', 'request_id': 'req-0911471f-1af9-4796-b6ee-d64889372e00'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:58.974995 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-0911471f-1af9-4796-b6ee-d64889372e00 admin admin] The token cache is being invalidate because project 25e7a4bfab9844b79ac0430d7807cbb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:01:58.975322 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-0911471f-1af9-4796-b6ee-d64889372e00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:58.976368 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-0911471f-1af9-4796-b6ee-d64889372e00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:58.977963 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1320/2649] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 15:01:58 2022] DELETE /identity/v3/projects/25e7a4bfab9844b79ac0430d7807cbb2 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:01:58.984866 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc85d08d-f0b2-422e-91da-3a59f28158ae None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:58.985626 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc85d08d-f0b2-422e-91da-3a59f28158ae None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:58.985626 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc85d08d-f0b2-422e-91da-3a59f28158ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:59.008065 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-fc85d08d-f0b2-422e-91da-3a59f28158ae None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:01:59.009342 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-fc85d08d-f0b2-422e-91da-3a59f28158ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:59.108735 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1330/2650] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:01:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:01:59.117930 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba35695-7029-4143-bdcc-567e9d0324c4 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:59.118297 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba35695-7029-4143-bdcc-567e9d0324c4 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:59.118486 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba35695-7029-4143-bdcc-567e9d0324c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:59.138886 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-2ba35695-7029-4143-bdcc-567e9d0324c4 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:01:59.140172 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-2ba35695-7029-4143-bdcc-567e9d0324c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:59.177060 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1321/2651] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:01:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:01:59.181762 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc85d08d-f0b2-422e-91da-3a59f28158ae None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:59.217993 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc85d08d-f0b2-422e-91da-3a59f28158ae None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:59.225412 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e66c00-1941-4969-ba34-3cfc9e59812c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:59.227737 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e66c00-1941-4969-ba34-3cfc9e59812c admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:59.230477 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e66c00-1941-4969-ba34-3cfc9e59812c admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:59.230477 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e66c00-1941-4969-ba34-3cfc9e59812c admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:59.230477 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e66c00-1941-4969-ba34-3cfc9e59812c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:01:59.230477 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e66c00-1941-4969-ba34-3cfc9e59812c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:59.231182 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e66c00-1941-4969-ba34-3cfc9e59812c admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:59.246323 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1331/2652] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jun 15 15:01:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:01:59.251169 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba35695-7029-4143-bdcc-567e9d0324c4 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:59.274167 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba35695-7029-4143-bdcc-567e9d0324c4 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:59.279509 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c42ac7-9a6c-4efa-b7c7-df47629aa838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:59.280450 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c42ac7-9a6c-4efa-b7c7-df47629aa838 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:59.280737 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c42ac7-9a6c-4efa-b7c7-df47629aa838 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:59.281016 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c42ac7-9a6c-4efa-b7c7-df47629aa838 admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:59.281722 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c42ac7-9a6c-4efa-b7c7-df47629aa838 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:59.283652 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c42ac7-9a6c-4efa-b7c7-df47629aa838 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:59.298551 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:01:59.375836 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1322/2653] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 15:01:59 2022] POST /identity/v3/projects => generated 373 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:01:59.381924 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e66c00-1941-4969-ba34-3cfc9e59812c admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:59.392404 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e66c00-1941-4969-ba34-3cfc9e59812c admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:59.440636 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:01:59.459342 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f609eb0-0c6a-4290-a50e-07668af75bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:59.462947 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f609eb0-0c6a-4290-a50e-07668af75bfb admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:59.462947 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f609eb0-0c6a-4290-a50e-07668af75bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:59.462947 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f609eb0-0c6a-4290-a50e-07668af75bfb admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:59.462947 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f609eb0-0c6a-4290-a50e-07668af75bfb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:59.463521 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f609eb0-0c6a-4290-a50e-07668af75bfb admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:59.573721 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1332/2654] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 15:01:59 2022] POST /identity/v3/users => generated 287 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:01:59.578012 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c42ac7-9a6c-4efa-b7c7-df47629aa838 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:59.589412 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c42ac7-9a6c-4efa-b7c7-df47629aa838 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:59.594079 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a398ac-00b7-42bc-9acc-26388a0dacbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:59.594756 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a398ac-00b7-42bc-9acc-26388a0dacbf admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:59.595037 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a398ac-00b7-42bc-9acc-26388a0dacbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:59.595311 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a398ac-00b7-42bc-9acc-26388a0dacbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:59.596040 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a398ac-00b7-42bc-9acc-26388a0dacbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:59.597724 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a398ac-00b7-42bc-9acc-26388a0dacbf admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:59.610181 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1323/2655] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 15:01:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:01:59.625278 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f609eb0-0c6a-4290-a50e-07668af75bfb admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:59.636837 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f609eb0-0c6a-4290-a50e-07668af75bfb admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:59.642248 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffdf8ab-8e2b-41bd-b603-e8f2b0de8a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:59.642918 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffdf8ab-8e2b-41bd-b603-e8f2b0de8a8b admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:59.643205 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffdf8ab-8e2b-41bd-b603-e8f2b0de8a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:59.643506 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffdf8ab-8e2b-41bd-b603-e8f2b0de8a8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:59.644283 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffdf8ab-8e2b-41bd-b603-e8f2b0de8a8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:59.646981 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffdf8ab-8e2b-41bd-b603-e8f2b0de8a8b admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:59.657711 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1333/2656] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 15:01:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:01:59.662555 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a398ac-00b7-42bc-9acc-26388a0dacbf admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:59.672670 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a398ac-00b7-42bc-9acc-26388a0dacbf admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:59.677341 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1011c96e-5d3b-4421-9a5c-6577caad052e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:59.678240 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1011c96e-5d3b-4421-9a5c-6577caad052e admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:59.678523 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1011c96e-5d3b-4421-9a5c-6577caad052e admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:59.678801 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1011c96e-5d3b-4421-9a5c-6577caad052e admin admin] PATH_INFO: `/v3/projects/2aeecf8eb64346e68a4038d37e1f0f2b/users/54d1fa49c0da487298898008a4b5afe7/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:59.695230 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1011c96e-5d3b-4421-9a5c-6577caad052e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2aeecf8eb64346e68a4038d37e1f0f2b, user_id=54d1fa49c0da487298898008a4b5afe7, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:59.697901 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1011c96e-5d3b-4421-9a5c-6577caad052e admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:59.801635 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1324/2657] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 15:01:59 2022] PUT /identity/v3/projects/2aeecf8eb64346e68a4038d37e1f0f2b/users/54d1fa49c0da487298898008a4b5afe7/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:01:59.818268 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22008000-15ed-429c-b772-8ae9dd5c58a2 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:59.818571 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22008000-15ed-429c-b772-8ae9dd5c58a2 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:59.818885 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22008000-15ed-429c-b772-8ae9dd5c58a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:59.847562 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-22008000-15ed-429c-b772-8ae9dd5c58a2 None None] MFA Rules not processed for user `54d1fa49c0da487298898008a4b5afe7`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:01:59.849134 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-22008000-15ed-429c-b772-8ae9dd5c58a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:59.917583 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1334/2658] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:01:59 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:01:59.929750 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1011c96e-5d3b-4421-9a5c-6577caad052e admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:59.934255 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-1011c96e-5d3b-4421-9a5c-6577caad052e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:00.010328 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1011c96e-5d3b-4421-9a5c-6577caad052e admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:00.015960 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f700ff6-fc8f-45b9-acd1-5b422563e3cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0927c05c340c48e48b18754888aa9c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:00.016649 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f700ff6-fc8f-45b9-acd1-5b422563e3cb service nova] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:00.016933 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f700ff6-fc8f-45b9-acd1-5b422563e3cb service nova] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:00.017216 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f700ff6-fc8f-45b9-acd1-5b422563e3cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:00.026590 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f700ff6-fc8f-45b9-acd1-5b422563e3cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:00.028183 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f700ff6-fc8f-45b9-acd1-5b422563e3cb service nova] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:00.037687 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1325/2659] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:01:59 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:02:00.298198 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22008000-15ed-429c-b772-8ae9dd5c58a2 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:00.369610 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22008000-15ed-429c-b772-8ae9dd5c58a2 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:00.374715 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6383d3bb-a70a-4694-b791-8e3d4d32f073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:00.375598 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6383d3bb-a70a-4694-b791-8e3d4d32f073 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:00.375875 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6383d3bb-a70a-4694-b791-8e3d4d32f073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:00.376254 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6383d3bb-a70a-4694-b791-8e3d4d32f073 admin admin] PATH_INFO: `/v3/users/54d1fa49c0da487298898008a4b5afe7` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:00.387342 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6383d3bb-a70a-4694-b791-8e3d4d32f073 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54d1fa49c0da487298898008a4b5afe7)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:00.387342 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6383d3bb-a70a-4694-b791-8e3d4d32f073 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:00.574333 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-6383d3bb-a70a-4694-b791-8e3d4d32f073 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54d1fa49c0da487298898008a4b5afe7'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:00.574991 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-6383d3bb-a70a-4694-b791-8e3d4d32f073 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54d1fa49c0da487298898008a4b5afe7'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:00.575613 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-6383d3bb-a70a-4694-b791-8e3d4d32f073 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54d1fa49c0da487298898008a4b5afe7'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:00.655638 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-6383d3bb-a70a-4694-b791-8e3d4d32f073 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54d1fa49c0da487298898008a4b5afe7'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:00.659694 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:00.671201 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-6383d3bb-a70a-4694-b791-8e3d4d32f073 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54d1fa49c0da487298898008a4b5afe7'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:00.684565 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1335/2660] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 15:02:00 2022] DELETE /identity/v3/users/54d1fa49c0da487298898008a4b5afe7 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:02:00.691334 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0d766d-32d5-4680-8551-66a246628068 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:00.691625 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0d766d-32d5-4680-8551-66a246628068 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:00.691933 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0d766d-32d5-4680-8551-66a246628068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:00.709473 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-9c0d766d-32d5-4680-8551-66a246628068 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:00.710696 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-9c0d766d-32d5-4680-8551-66a246628068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:00.775106 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1326/2661] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:02:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:02:00.786269 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6383d3bb-a70a-4694-b791-8e3d4d32f073 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:00.787858 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-6383d3bb-a70a-4694-b791-8e3d4d32f073 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:00.850374 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6383d3bb-a70a-4694-b791-8e3d4d32f073 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:00.854120 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c1c2b0-84bd-4fcd-8d9e-0acde7fa6e88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:00.854770 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c1c2b0-84bd-4fcd-8d9e-0acde7fa6e88 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:00.855053 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c1c2b0-84bd-4fcd-8d9e-0acde7fa6e88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:00.855332 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c1c2b0-84bd-4fcd-8d9e-0acde7fa6e88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:00.865553 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c1c2b0-84bd-4fcd-8d9e-0acde7fa6e88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:00.866986 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c1c2b0-84bd-4fcd-8d9e-0acde7fa6e88 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:00.877320 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1336/2662] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:02:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:02:01.262107 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba998afe-1235-4b17-8f22-9420d84ede58 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:01.262646 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba998afe-1235-4b17-8f22-9420d84ede58 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:01.262764 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba998afe-1235-4b17-8f22-9420d84ede58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:01.280459 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-ba998afe-1235-4b17-8f22-9420d84ede58 None None] MFA Rules not processed for user `e209ae2f7fa7412b95c7c43c74c14426`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:01.281651 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-ba998afe-1235-4b17-8f22-9420d84ede58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:01.312222 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c1c2b0-84bd-4fcd-8d9e-0acde7fa6e88 service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:01.313770 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e6c1c2b0-84bd-4fcd-8d9e-0acde7fa6e88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:01.328406 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c1c2b0-84bd-4fcd-8d9e-0acde7fa6e88 service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:01.334091 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4fb7c3-b90c-4b13-8e0b-a50b5eee3f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:01.335156 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4fb7c3-b90c-4b13-8e0b-a50b5eee3f09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:01.335372 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:01.335581 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4fb7c3-b90c-4b13-8e0b-a50b5eee3f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:01.335869 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4fb7c3-b90c-4b13-8e0b-a50b5eee3f09 admin admin] PATH_INFO: `/v3/projects/2aeecf8eb64346e68a4038d37e1f0f2b` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:01.337181 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4fb7c3-b90c-4b13-8e0b-a50b5eee3f09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2aeecf8eb64346e68a4038d37e1f0f2b)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:01.339161 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4fb7c3-b90c-4b13-8e0b-a50b5eee3f09 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:01.353701 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1327/2663] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:02:01 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:02:01.481000 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-4d4fb7c3-b90c-4b13-8e0b-a50b5eee3f09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2aeecf8eb64346e68a4038d37e1f0f2b', 'request_id': 'req-4d4fb7c3-b90c-4b13-8e0b-a50b5eee3f09'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:01.491617 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-4d4fb7c3-b90c-4b13-8e0b-a50b5eee3f09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2aeecf8eb64346e68a4038d37e1f0f2b', 'request_id': 'req-4d4fb7c3-b90c-4b13-8e0b-a50b5eee3f09'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:01.571814 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-4d4fb7c3-b90c-4b13-8e0b-a50b5eee3f09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2aeecf8eb64346e68a4038d37e1f0f2b', 'request_id': 'req-4d4fb7c3-b90c-4b13-8e0b-a50b5eee3f09'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:01.583683 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-4d4fb7c3-b90c-4b13-8e0b-a50b5eee3f09 admin admin] The token cache is being invalidate because project 2aeecf8eb64346e68a4038d37e1f0f2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:02:01.584001 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-4d4fb7c3-b90c-4b13-8e0b-a50b5eee3f09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:01.584962 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-4d4fb7c3-b90c-4b13-8e0b-a50b5eee3f09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:01.586531 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1337/2664] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 15:02:01 2022] DELETE /identity/v3/projects/2aeecf8eb64346e68a4038d37e1f0f2b => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:02:01.591405 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc5277-2271-495e-8a36-1bb1b0851299 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:01.591697 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc5277-2271-495e-8a36-1bb1b0851299 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:01.591986 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc5277-2271-495e-8a36-1bb1b0851299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:01.612002 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-63dc5277-2271-495e-8a36-1bb1b0851299 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:01.613370 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-63dc5277-2271-495e-8a36-1bb1b0851299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:01.678882 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1328/2665] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:02:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:02:01.687522 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c9dbcd-61a4-40b5-9583-1dfe0fbea75e None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:01.688076 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c9dbcd-61a4-40b5-9583-1dfe0fbea75e None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:01.688473 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c9dbcd-61a4-40b5-9583-1dfe0fbea75e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:01.707915 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-69c9dbcd-61a4-40b5-9583-1dfe0fbea75e None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:01.709158 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-69c9dbcd-61a4-40b5-9583-1dfe0fbea75e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:01.717671 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1338/2666] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:02:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:02:01.722100 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc5277-2271-495e-8a36-1bb1b0851299 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:01.733620 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc5277-2271-495e-8a36-1bb1b0851299 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:01.738553 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822db4e2-5cb6-4895-93d3-fcea0e2ee488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:01.739788 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822db4e2-5cb6-4895-93d3-fcea0e2ee488 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:01.740070 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822db4e2-5cb6-4895-93d3-fcea0e2ee488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:01.740506 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822db4e2-5cb6-4895-93d3-fcea0e2ee488 admin admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:01.741235 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822db4e2-5cb6-4895-93d3-fcea0e2ee488 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:02:01.741697 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822db4e2-5cb6-4895-93d3-fcea0e2ee488 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:01.743295 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822db4e2-5cb6-4895-93d3-fcea0e2ee488 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:01.757066 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1329/2667] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jun 15 15:02:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:02:01.761513 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c9dbcd-61a4-40b5-9583-1dfe0fbea75e None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:01.772388 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c9dbcd-61a4-40b5-9583-1dfe0fbea75e None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:01.777556 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b02579-bac5-4442-b053-18cafd7e034a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:01.778241 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b02579-bac5-4442-b053-18cafd7e034a admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:01.778512 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b02579-bac5-4442-b053-18cafd7e034a admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:01.778792 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b02579-bac5-4442-b053-18cafd7e034a admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:01.779748 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b02579-bac5-4442-b053-18cafd7e034a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:01.781750 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b02579-bac5-4442-b053-18cafd7e034a admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:01.897081 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1339/2668] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 15:02:01 2022] POST /identity/v3/projects => generated 363 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:02:01.898300 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822db4e2-5cb6-4895-93d3-fcea0e2ee488 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:01.910415 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822db4e2-5cb6-4895-93d3-fcea0e2ee488 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:01.979937 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72e3d0f-fff6-437f-9c17-852b087faebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:01.980925 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72e3d0f-fff6-437f-9c17-852b087faebe admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:01.981229 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72e3d0f-fff6-437f-9c17-852b087faebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:01.984010 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72e3d0f-fff6-437f-9c17-852b087faebe admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:01.984010 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72e3d0f-fff6-437f-9c17-852b087faebe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:01.984243 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72e3d0f-fff6-437f-9c17-852b087faebe admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:02.066538 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1330/2669] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 15:02:01 2022] POST /identity/v3/users => generated 282 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:02:02.071241 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b02579-bac5-4442-b053-18cafd7e034a admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:02.082964 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b02579-bac5-4442-b053-18cafd7e034a admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:02.088254 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15e69db-6aa2-47a3-bbfc-b1238f7693ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:02.088940 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e69db-6aa2-47a3-bbfc-b1238f7693ec admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:02.089233 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e69db-6aa2-47a3-bbfc-b1238f7693ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:02.089518 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e69db-6aa2-47a3-bbfc-b1238f7693ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:02.090250 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15e69db-6aa2-47a3-bbfc-b1238f7693ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:02.091845 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15e69db-6aa2-47a3-bbfc-b1238f7693ec admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:02.103861 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1340/2670] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 15:02:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:02:02.107999 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72e3d0f-fff6-437f-9c17-852b087faebe admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:02.118951 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72e3d0f-fff6-437f-9c17-852b087faebe admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:02.123669 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a3bb22-d5ea-407e-b642-48af8b1758c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:02.124618 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a3bb22-d5ea-407e-b642-48af8b1758c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:02.124898 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a3bb22-d5ea-407e-b642-48af8b1758c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:02.125266 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a3bb22-d5ea-407e-b642-48af8b1758c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:02.125990 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a3bb22-d5ea-407e-b642-48af8b1758c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:02.127704 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a3bb22-d5ea-407e-b642-48af8b1758c2 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:02.138845 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1331/2671] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 15:02:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:02:02.143164 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15e69db-6aa2-47a3-bbfc-b1238f7693ec admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:02.153267 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15e69db-6aa2-47a3-bbfc-b1238f7693ec admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:02.158161 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af396061-294a-43ae-bb26-5ea19b2d428b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:02.161245 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af396061-294a-43ae-bb26-5ea19b2d428b admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:02.161245 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af396061-294a-43ae-bb26-5ea19b2d428b admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:02.161245 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af396061-294a-43ae-bb26-5ea19b2d428b admin admin] PATH_INFO: `/v3/projects/f8afdbd511ef41e4afbd6577e45eb12b/users/fde16a2d102645679b31fc871e533a2b/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:02.175020 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af396061-294a-43ae-bb26-5ea19b2d428b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8afdbd511ef41e4afbd6577e45eb12b, user_id=fde16a2d102645679b31fc871e533a2b, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:02.178577 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af396061-294a-43ae-bb26-5ea19b2d428b admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:02.181763 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:02.247904 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1341/2672] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 15:02:02 2022] PUT /identity/v3/projects/f8afdbd511ef41e4afbd6577e45eb12b/users/fde16a2d102645679b31fc871e533a2b/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:02:02.259955 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ec9a9c-c7f0-4a5b-84c5-ff198c7ab655 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:02.260269 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ec9a9c-c7f0-4a5b-84c5-ff198c7ab655 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:02.260604 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ec9a9c-c7f0-4a5b-84c5-ff198c7ab655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:02.291668 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-d9ec9a9c-c7f0-4a5b-84c5-ff198c7ab655 None None] MFA Rules not processed for user `fde16a2d102645679b31fc871e533a2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:02.292967 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-d9ec9a9c-c7f0-4a5b-84c5-ff198c7ab655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:02.341320 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:02.363179 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1332/2673] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:02:02 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:02:02.368259 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af396061-294a-43ae-bb26-5ea19b2d428b admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:02.434025 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af396061-294a-43ae-bb26-5ea19b2d428b admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:02.439115 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c2e3de-61b4-4209-9456-86bfb29ae750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:02.439787 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c2e3de-61b4-4209-9456-86bfb29ae750 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:02.440073 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c2e3de-61b4-4209-9456-86bfb29ae750 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:02.440394 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c2e3de-61b4-4209-9456-86bfb29ae750 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:02.441090 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c2e3de-61b4-4209-9456-86bfb29ae750 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:02.443476 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c2e3de-61b4-4209-9456-86bfb29ae750 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:02.589286 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1342/2674] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 15:02:02 2022] POST /identity/v3/projects => generated 361 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:02:02.591054 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ec9a9c-c7f0-4a5b-84c5-ff198c7ab655 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:02.603824 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ec9a9c-c7f0-4a5b-84c5-ff198c7ab655 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:02.665227 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea467995-1f4b-4046-b522-4129b3ab3c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:02.665908 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea467995-1f4b-4046-b522-4129b3ab3c07 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:02.666191 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea467995-1f4b-4046-b522-4129b3ab3c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:02.666468 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea467995-1f4b-4046-b522-4129b3ab3c07 admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:02.667235 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea467995-1f4b-4046-b522-4129b3ab3c07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:02.669202 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea467995-1f4b-4046-b522-4129b3ab3c07 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:02.765200 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1333/2675] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 15:02:02 2022] POST /identity/v3/users => generated 287 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:02:02.766554 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c2e3de-61b4-4209-9456-86bfb29ae750 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:02.777321 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c2e3de-61b4-4209-9456-86bfb29ae750 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:02.782174 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93153083-3286-44cc-9cb3-0f8990ae0480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:02.782860 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93153083-3286-44cc-9cb3-0f8990ae0480 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:02.783157 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93153083-3286-44cc-9cb3-0f8990ae0480 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:02.783442 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93153083-3286-44cc-9cb3-0f8990ae0480 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:02.784325 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93153083-3286-44cc-9cb3-0f8990ae0480 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:02.800130 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93153083-3286-44cc-9cb3-0f8990ae0480 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:02.800130 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1343/2676] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 15:02:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:02:02.803975 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea467995-1f4b-4046-b522-4129b3ab3c07 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:02.814442 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea467995-1f4b-4046-b522-4129b3ab3c07 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:02.819366 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3732eb-d578-488d-80e1-1e1d4cbf7bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:02.820276 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3732eb-d578-488d-80e1-1e1d4cbf7bc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:02.820563 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3732eb-d578-488d-80e1-1e1d4cbf7bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:02.820838 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3732eb-d578-488d-80e1-1e1d4cbf7bc5 admin admin] PATH_INFO: `/v3/projects/5df28325791644b3a65890154ce9effc/users/df0ebacacfb4464b8b469f7ef7a89eb8/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:02.840046 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3732eb-d578-488d-80e1-1e1d4cbf7bc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5df28325791644b3a65890154ce9effc, user_id=df0ebacacfb4464b8b469f7ef7a89eb8, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:02.843881 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3732eb-d578-488d-80e1-1e1d4cbf7bc5 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:02.951082 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1334/2677] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 15:02:02 2022] PUT /identity/v3/projects/5df28325791644b3a65890154ce9effc/users/df0ebacacfb4464b8b469f7ef7a89eb8/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:02:02.954993 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93153083-3286-44cc-9cb3-0f8990ae0480 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:02.966091 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93153083-3286-44cc-9cb3-0f8990ae0480 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:03.037717 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6866d7d-2d57-48a9-9818-56e4008e110c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:03.038407 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6866d7d-2d57-48a9-9818-56e4008e110c admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:03.038691 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6866d7d-2d57-48a9-9818-56e4008e110c admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:03.038980 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6866d7d-2d57-48a9-9818-56e4008e110c admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:03.039700 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6866d7d-2d57-48a9-9818-56e4008e110c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:03.041605 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6866d7d-2d57-48a9-9818-56e4008e110c admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:03.054904 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1344/2678] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 15:02:02 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:02:03.059183 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3732eb-d578-488d-80e1-1e1d4cbf7bc5 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:03.070598 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3732eb-d578-488d-80e1-1e1d4cbf7bc5 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:03.075631 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04a1e18-ce8c-43a1-93b7-9f6d196cf636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:03.077935 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04a1e18-ce8c-43a1-93b7-9f6d196cf636 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:03.077935 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04a1e18-ce8c-43a1-93b7-9f6d196cf636 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:03.077935 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04a1e18-ce8c-43a1-93b7-9f6d196cf636 admin admin] PATH_INFO: `/v3/projects/5df28325791644b3a65890154ce9effc/users/df0ebacacfb4464b8b469f7ef7a89eb8/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:03.079240 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04a1e18-ce8c-43a1-93b7-9f6d196cf636 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5df28325791644b3a65890154ce9effc, user_id=df0ebacacfb4464b8b469f7ef7a89eb8, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:03.082076 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04a1e18-ce8c-43a1-93b7-9f6d196cf636 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:03.094038 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1335/2679] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 15:02:03 2022] PUT /identity/v3/projects/5df28325791644b3a65890154ce9effc/users/df0ebacacfb4464b8b469f7ef7a89eb8/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:02:03.106167 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e3abc6-cc34-47fd-883a-ab43adcf8853 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:03.106522 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e3abc6-cc34-47fd-883a-ab43adcf8853 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:03.106750 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e3abc6-cc34-47fd-883a-ab43adcf8853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:03.136466 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-00e3abc6-cc34-47fd-883a-ab43adcf8853 None None] MFA Rules not processed for user `df0ebacacfb4464b8b469f7ef7a89eb8`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:03.137690 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-00e3abc6-cc34-47fd-883a-ab43adcf8853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:03.194728 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:03.213520 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1345/2680] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:02:03 2022] POST /identity/v3/auth/tokens => generated 3295 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:02:03.225714 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04a1e18-ce8c-43a1-93b7-9f6d196cf636 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:03.227434 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-f04a1e18-ce8c-43a1-93b7-9f6d196cf636 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:03.289203 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04a1e18-ce8c-43a1-93b7-9f6d196cf636 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:03.293393 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b8c66e-fe10-47a5-839e-750f3d9a36b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:03.294404 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b8c66e-fe10-47a5-839e-750f3d9a36b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:03.294404 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b8c66e-fe10-47a5-839e-750f3d9a36b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:03.294404 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b8c66e-fe10-47a5-839e-750f3d9a36b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:03.305062 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b8c66e-fe10-47a5-839e-750f3d9a36b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:03.306515 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b8c66e-fe10-47a5-839e-750f3d9a36b4 service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:03.317183 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1336/2681] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:02:03 2022] GET /identity/v3/auth/tokens => generated 3295 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:02:03.528544 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e3abc6-cc34-47fd-883a-ab43adcf8853 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:03.539508 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e3abc6-cc34-47fd-883a-ab43adcf8853 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:03.543567 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec246d5-3826-47ff-86f0-cbe2a1d5ffa4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:03.544286 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec246d5-3826-47ff-86f0-cbe2a1d5ffa4 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:03.544548 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec246d5-3826-47ff-86f0-cbe2a1d5ffa4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:03.544821 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec246d5-3826-47ff-86f0-cbe2a1d5ffa4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:03.613025 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec246d5-3826-47ff-86f0-cbe2a1d5ffa4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:03.615039 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec246d5-3826-47ff-86f0-cbe2a1d5ffa4 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:03.625369 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1346/2682] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:02:03 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:02:13.312353 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=77,keystone:INSERT=5 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:13.620594 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=32 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:18.083384 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b8c66e-fe10-47a5-839e-750f3d9a36b4 service cinder] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:18.148870 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b8c66e-fe10-47a5-839e-750f3d9a36b4 service cinder] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:18.153454 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e88e088-d24a-4109-ba50-3ecd0e51c9db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:18.154354 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e88e088-d24a-4109-ba50-3ecd0e51c9db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:18.154637 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e88e088-d24a-4109-ba50-3ecd0e51c9db admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:18.154911 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e88e088-d24a-4109-ba50-3ecd0e51c9db admin admin] PATH_INFO: `/v3/users/fde16a2d102645679b31fc871e533a2b` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:18.156109 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e88e088-d24a-4109-ba50-3ecd0e51c9db admin admin] RBAC: Authorizing `identity:delete_user(user_id=fde16a2d102645679b31fc871e533a2b)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:18.158066 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e88e088-d24a-4109-ba50-3ecd0e51c9db admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:18.362581 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-4e88e088-d24a-4109-ba50-3ecd0e51c9db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fde16a2d102645679b31fc871e533a2b'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:18.363233 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-4e88e088-d24a-4109-ba50-3ecd0e51c9db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fde16a2d102645679b31fc871e533a2b'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:18.371872 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-4e88e088-d24a-4109-ba50-3ecd0e51c9db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fde16a2d102645679b31fc871e533a2b'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:18.372711 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-4e88e088-d24a-4109-ba50-3ecd0e51c9db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fde16a2d102645679b31fc871e533a2b'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:18.463098 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-4e88e088-d24a-4109-ba50-3ecd0e51c9db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fde16a2d102645679b31fc871e533a2b'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:18.481459 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1337/2683] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 15:02:18 2022] DELETE /identity/v3/users/fde16a2d102645679b31fc871e533a2b => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:02:18.486213 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39fafd4-87be-4ea9-84a6-347aab7c0ba5 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:18.491381 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39fafd4-87be-4ea9-84a6-347aab7c0ba5 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:18.491381 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39fafd4-87be-4ea9-84a6-347aab7c0ba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:18.515100 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-d39fafd4-87be-4ea9-84a6-347aab7c0ba5 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:18.516502 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-d39fafd4-87be-4ea9-84a6-347aab7c0ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:18.583689 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1347/2684] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:02:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:02:18.595445 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e88e088-d24a-4109-ba50-3ecd0e51c9db admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:18.600513 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-4e88e088-d24a-4109-ba50-3ecd0e51c9db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:18.674434 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e88e088-d24a-4109-ba50-3ecd0e51c9db admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:18.678991 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13db4f9d-3caf-4e5b-ad2b-821e08721634 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:18.679716 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13db4f9d-3caf-4e5b-ad2b-821e08721634 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:18.680001 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13db4f9d-3caf-4e5b-ad2b-821e08721634 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:18.680311 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13db4f9d-3caf-4e5b-ad2b-821e08721634 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:18.690799 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13db4f9d-3caf-4e5b-ad2b-821e08721634 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:18.692353 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13db4f9d-3caf-4e5b-ad2b-821e08721634 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:18.703366 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1338/2685] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:02:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:02:19.168556 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39fafd4-87be-4ea9-84a6-347aab7c0ba5 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:19.170103 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-d39fafd4-87be-4ea9-84a6-347aab7c0ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:19.183810 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39fafd4-87be-4ea9-84a6-347aab7c0ba5 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:19.190816 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f3a701-3126-4e74-b48c-f135e36b0d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:19.191828 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f3a701-3126-4e74-b48c-f135e36b0d2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:19.192129 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f3a701-3126-4e74-b48c-f135e36b0d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:19.192534 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f3a701-3126-4e74-b48c-f135e36b0d2d admin admin] PATH_INFO: `/v3/projects/f8afdbd511ef41e4afbd6577e45eb12b` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:19.194523 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f3a701-3126-4e74-b48c-f135e36b0d2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8afdbd511ef41e4afbd6577e45eb12b)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:19.196521 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f3a701-3126-4e74-b48c-f135e36b0d2d admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:19.338680 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-88f3a701-3126-4e74-b48c-f135e36b0d2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8afdbd511ef41e4afbd6577e45eb12b', 'request_id': 'req-88f3a701-3126-4e74-b48c-f135e36b0d2d'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:19.345031 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-88f3a701-3126-4e74-b48c-f135e36b0d2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8afdbd511ef41e4afbd6577e45eb12b', 'request_id': 'req-88f3a701-3126-4e74-b48c-f135e36b0d2d'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:19.422999 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-88f3a701-3126-4e74-b48c-f135e36b0d2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8afdbd511ef41e4afbd6577e45eb12b', 'request_id': 'req-88f3a701-3126-4e74-b48c-f135e36b0d2d'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:19.434992 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-88f3a701-3126-4e74-b48c-f135e36b0d2d admin admin] The token cache is being invalidate because project f8afdbd511ef41e4afbd6577e45eb12b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:02:19.435280 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-88f3a701-3126-4e74-b48c-f135e36b0d2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:19.435914 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-88f3a701-3126-4e74-b48c-f135e36b0d2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:19.437495 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1348/2686] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 15:02:19 2022] DELETE /identity/v3/projects/f8afdbd511ef41e4afbd6577e45eb12b => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:02:19.441534 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13db4f9d-3caf-4e5b-ad2b-821e08721634 service neutron] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:19.451301 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13db4f9d-3caf-4e5b-ad2b-821e08721634 service neutron] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:19.460683 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:19.508221 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499ed9ea-73be-43af-ab31-b607747a3479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:19.509120 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499ed9ea-73be-43af-ab31-b607747a3479 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:19.509407 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499ed9ea-73be-43af-ab31-b607747a3479 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:19.509675 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499ed9ea-73be-43af-ab31-b607747a3479 admin admin] PATH_INFO: `/v3/users/df0ebacacfb4464b8b469f7ef7a89eb8` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:19.510843 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499ed9ea-73be-43af-ab31-b607747a3479 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df0ebacacfb4464b8b469f7ef7a89eb8)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:19.512804 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499ed9ea-73be-43af-ab31-b607747a3479 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:19.728268 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-499ed9ea-73be-43af-ab31-b607747a3479 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df0ebacacfb4464b8b469f7ef7a89eb8'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:19.728845 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-499ed9ea-73be-43af-ab31-b607747a3479 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df0ebacacfb4464b8b469f7ef7a89eb8'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:19.736889 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-499ed9ea-73be-43af-ab31-b607747a3479 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df0ebacacfb4464b8b469f7ef7a89eb8'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:19.737561 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-499ed9ea-73be-43af-ab31-b607747a3479 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df0ebacacfb4464b8b469f7ef7a89eb8'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:19.805661 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-499ed9ea-73be-43af-ab31-b607747a3479 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df0ebacacfb4464b8b469f7ef7a89eb8'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:19.829069 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1339/2687] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 15:02:19 2022] DELETE /identity/v3/users/df0ebacacfb4464b8b469f7ef7a89eb8 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:02:20.264115 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f3a701-3126-4e74-b48c-f135e36b0d2d admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:20.265552 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-88f3a701-3126-4e74-b48c-f135e36b0d2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:20.278522 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f3a701-3126-4e74-b48c-f135e36b0d2d admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:20.336804 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c18f711-6831-4eeb-b82d-2ea0ce547878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:20.337640 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c18f711-6831-4eeb-b82d-2ea0ce547878 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:20.337938 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c18f711-6831-4eeb-b82d-2ea0ce547878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:20.338283 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c18f711-6831-4eeb-b82d-2ea0ce547878 admin admin] PATH_INFO: `/v3/projects/5df28325791644b3a65890154ce9effc` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:20.339557 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c18f711-6831-4eeb-b82d-2ea0ce547878 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5df28325791644b3a65890154ce9effc)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:20.341504 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c18f711-6831-4eeb-b82d-2ea0ce547878 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:20.363554 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:20.463868 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5c18f711-6831-4eeb-b82d-2ea0ce547878 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5df28325791644b3a65890154ce9effc', 'request_id': 'req-5c18f711-6831-4eeb-b82d-2ea0ce547878'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:20.470013 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5c18f711-6831-4eeb-b82d-2ea0ce547878 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5df28325791644b3a65890154ce9effc', 'request_id': 'req-5c18f711-6831-4eeb-b82d-2ea0ce547878'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:20.565937 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5c18f711-6831-4eeb-b82d-2ea0ce547878 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5df28325791644b3a65890154ce9effc', 'request_id': 'req-5c18f711-6831-4eeb-b82d-2ea0ce547878'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:20.577371 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5c18f711-6831-4eeb-b82d-2ea0ce547878 admin admin] The token cache is being invalidate because project 5df28325791644b3a65890154ce9effc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:02:20.577657 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5c18f711-6831-4eeb-b82d-2ea0ce547878 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:20.578278 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5c18f711-6831-4eeb-b82d-2ea0ce547878 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:20.579851 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1349/2688] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 15:02:20 2022] DELETE /identity/v3/projects/5df28325791644b3a65890154ce9effc => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:02:20.584929 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeaead4-811b-4b5c-ac22-a19b307c38aa None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:20.585619 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeaead4-811b-4b5c-ac22-a19b307c38aa None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:20.585619 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeaead4-811b-4b5c-ac22-a19b307c38aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:20.638787 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-aaeaead4-811b-4b5c-ac22-a19b307c38aa None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:20.640072 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-aaeaead4-811b-4b5c-ac22-a19b307c38aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:20.700607 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1340/2689] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:02:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:02:20.708536 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b121bfe-af97-4baa-ba59-2e3db3c98a07 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:20.708829 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b121bfe-af97-4baa-ba59-2e3db3c98a07 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:20.709113 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b121bfe-af97-4baa-ba59-2e3db3c98a07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:20.728863 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-7b121bfe-af97-4baa-ba59-2e3db3c98a07 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:20.730071 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-7b121bfe-af97-4baa-ba59-2e3db3c98a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:20.738087 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1350/2690] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:02:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:02:20.742624 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeaead4-811b-4b5c-ac22-a19b307c38aa None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:20.753024 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeaead4-811b-4b5c-ac22-a19b307c38aa None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:20.757797 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d935e402-8e85-4689-b098-95b77c791ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:20.758550 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d935e402-8e85-4689-b098-95b77c791ca8 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:20.758835 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d935e402-8e85-4689-b098-95b77c791ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:20.759116 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d935e402-8e85-4689-b098-95b77c791ca8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:20.759840 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d935e402-8e85-4689-b098-95b77c791ca8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:02:20.760176 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d935e402-8e85-4689-b098-95b77c791ca8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:20.761768 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d935e402-8e85-4689-b098-95b77c791ca8 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:20.764338 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:20.774526 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1341/2691] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jun 15 15:02:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:02:20.778997 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b121bfe-af97-4baa-ba59-2e3db3c98a07 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:20.788684 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b121bfe-af97-4baa-ba59-2e3db3c98a07 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:20.793963 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a40d0d4-c424-4c59-bf0b-f99cb1c2771b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:20.795280 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a40d0d4-c424-4c59-bf0b-f99cb1c2771b admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:20.795280 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a40d0d4-c424-4c59-bf0b-f99cb1c2771b admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:20.795280 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a40d0d4-c424-4c59-bf0b-f99cb1c2771b admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:20.795903 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a40d0d4-c424-4c59-bf0b-f99cb1c2771b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:20.797908 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a40d0d4-c424-4c59-bf0b-f99cb1c2771b admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:20.870871 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1351/2692] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 15:02:20 2022] POST /identity/v3/projects => generated 385 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:02:20.875409 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d935e402-8e85-4689-b098-95b77c791ca8 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:20.885480 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d935e402-8e85-4689-b098-95b77c791ca8 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:20.943072 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec66b473-2cb4-4e63-9cb3-9668a9020c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:20.943736 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec66b473-2cb4-4e63-9cb3-9668a9020c94 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:20.944021 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec66b473-2cb4-4e63-9cb3-9668a9020c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:20.944328 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec66b473-2cb4-4e63-9cb3-9668a9020c94 admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:20.945009 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec66b473-2cb4-4e63-9cb3-9668a9020c94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:20.946912 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec66b473-2cb4-4e63-9cb3-9668a9020c94 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:21.006074 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1342/2693] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 15:02:20 2022] POST /identity/v3/users => generated 293 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:02:21.010320 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a40d0d4-c424-4c59-bf0b-f99cb1c2771b admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:21.021480 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a40d0d4-c424-4c59-bf0b-f99cb1c2771b admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:21.026724 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2802596b-5c59-456b-966b-0e8c4dea7698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:21.027411 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2802596b-5c59-456b-966b-0e8c4dea7698 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:21.027693 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2802596b-5c59-456b-966b-0e8c4dea7698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:21.028003 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2802596b-5c59-456b-966b-0e8c4dea7698 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:21.028738 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2802596b-5c59-456b-966b-0e8c4dea7698 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:21.030333 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2802596b-5c59-456b-966b-0e8c4dea7698 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:21.042683 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1352/2694] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 15:02:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:02:21.046715 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec66b473-2cb4-4e63-9cb3-9668a9020c94 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:21.056974 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec66b473-2cb4-4e63-9cb3-9668a9020c94 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:21.061696 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddd171f-a48c-4b9e-bcf1-17b03b3a00d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:21.062354 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddd171f-a48c-4b9e-bcf1-17b03b3a00d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:21.062641 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddd171f-a48c-4b9e-bcf1-17b03b3a00d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:21.062919 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddd171f-a48c-4b9e-bcf1-17b03b3a00d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:21.063640 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddd171f-a48c-4b9e-bcf1-17b03b3a00d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:21.065250 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddd171f-a48c-4b9e-bcf1-17b03b3a00d8 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:21.076311 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1343/2695] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 15:02:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:02:21.080635 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2802596b-5c59-456b-966b-0e8c4dea7698 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:21.091243 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2802596b-5c59-456b-966b-0e8c4dea7698 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:21.096059 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb44228-0c41-4f12-8f10-2060a1ed45ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:21.097043 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb44228-0c41-4f12-8f10-2060a1ed45ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:21.097324 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb44228-0c41-4f12-8f10-2060a1ed45ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:21.097616 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb44228-0c41-4f12-8f10-2060a1ed45ac admin admin] PATH_INFO: `/v3/projects/7fb15dd92fd0428caf31b9e9064a0c5b/users/b07d457a47194b0ca1a9de33fe15409e/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:21.113865 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb44228-0c41-4f12-8f10-2060a1ed45ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fb15dd92fd0428caf31b9e9064a0c5b, user_id=b07d457a47194b0ca1a9de33fe15409e, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:21.116660 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb44228-0c41-4f12-8f10-2060a1ed45ac admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:21.240652 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1353/2696] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 15:02:21 2022] PUT /identity/v3/projects/7fb15dd92fd0428caf31b9e9064a0c5b/users/b07d457a47194b0ca1a9de33fe15409e/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:02:21.253550 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e672f5-cea0-4270-bd78-27b9b547fc59 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:21.253890 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e672f5-cea0-4270-bd78-27b9b547fc59 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:21.254176 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e672f5-cea0-4270-bd78-27b9b547fc59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:21.284834 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-a7e672f5-cea0-4270-bd78-27b9b547fc59 None None] MFA Rules not processed for user `b07d457a47194b0ca1a9de33fe15409e`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:21.286261 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-a7e672f5-cea0-4270-bd78-27b9b547fc59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:21.360862 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1344/2697] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:02:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:02:21.373309 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb44228-0c41-4f12-8f10-2060a1ed45ac admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:21.375406 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-9fb44228-0c41-4f12-8f10-2060a1ed45ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:21.435930 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb44228-0c41-4f12-8f10-2060a1ed45ac admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:21.439598 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f464ef6-9de1-41c7-b611-2e338ea76402 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:21.440276 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f464ef6-9de1-41c7-b611-2e338ea76402 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:21.440562 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f464ef6-9de1-41c7-b611-2e338ea76402 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:21.440840 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f464ef6-9de1-41c7-b611-2e338ea76402 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:21.449322 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f464ef6-9de1-41c7-b611-2e338ea76402 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:21.455635 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f464ef6-9de1-41c7-b611-2e338ea76402 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:21.461638 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1354/2698] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:02:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:02:21.481739 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e672f5-cea0-4270-bd78-27b9b547fc59 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:21.546082 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e672f5-cea0-4270-bd78-27b9b547fc59 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:21.550739 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9b30c0-f5bf-4738-8e55-2f7af9ecc806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:21.551617 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b30c0-f5bf-4738-8e55-2f7af9ecc806 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:21.551902 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b30c0-f5bf-4738-8e55-2f7af9ecc806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:21.552210 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b30c0-f5bf-4738-8e55-2f7af9ecc806 admin admin] PATH_INFO: `/v3/users/b07d457a47194b0ca1a9de33fe15409e` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:21.553422 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9b30c0-f5bf-4738-8e55-2f7af9ecc806 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b07d457a47194b0ca1a9de33fe15409e)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:21.555966 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9b30c0-f5bf-4738-8e55-2f7af9ecc806 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:21.560988 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:21.773386 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-ca9b30c0-f5bf-4738-8e55-2f7af9ecc806 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b07d457a47194b0ca1a9de33fe15409e'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:21.774060 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-ca9b30c0-f5bf-4738-8e55-2f7af9ecc806 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b07d457a47194b0ca1a9de33fe15409e'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:21.781176 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-ca9b30c0-f5bf-4738-8e55-2f7af9ecc806 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b07d457a47194b0ca1a9de33fe15409e'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:21.781825 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-ca9b30c0-f5bf-4738-8e55-2f7af9ecc806 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b07d457a47194b0ca1a9de33fe15409e'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:21.857077 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-ca9b30c0-f5bf-4738-8e55-2f7af9ecc806 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b07d457a47194b0ca1a9de33fe15409e'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:21.876665 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1345/2699] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 15:02:21 2022] DELETE /identity/v3/users/b07d457a47194b0ca1a9de33fe15409e => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:02:21.882843 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd301394-562f-4149-a2a6-da742c1604b1 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:21.883472 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd301394-562f-4149-a2a6-da742c1604b1 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:21.883954 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd301394-562f-4149-a2a6-da742c1604b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:21.904290 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-bd301394-562f-4149-a2a6-da742c1604b1 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:21.911622 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-bd301394-562f-4149-a2a6-da742c1604b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:21.948138 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:21.965900 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1355/2700] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:02:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:02:21.975722 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9b30c0-f5bf-4738-8e55-2f7af9ecc806 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:21.977486 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-ca9b30c0-f5bf-4738-8e55-2f7af9ecc806 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:22.035950 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9b30c0-f5bf-4738-8e55-2f7af9ecc806 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:22.039740 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb1571-871a-4946-bb53-2cae5d544a4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:22.040415 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb1571-871a-4946-bb53-2cae5d544a4c service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:22.040693 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb1571-871a-4946-bb53-2cae5d544a4c service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:22.040963 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb1571-871a-4946-bb53-2cae5d544a4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:22.050116 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fb1571-871a-4946-bb53-2cae5d544a4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:22.051580 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fb1571-871a-4946-bb53-2cae5d544a4c service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:22.061445 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1346/2701] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:02:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 15:02:22.596949 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd301394-562f-4149-a2a6-da742c1604b1 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:22.598457 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-bd301394-562f-4149-a2a6-da742c1604b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:22.611193 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd301394-562f-4149-a2a6-da742c1604b1 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:22.616480 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95252ea-cd6f-4dd7-a652-763268875bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:22.618101 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95252ea-cd6f-4dd7-a652-763268875bba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:22.618389 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95252ea-cd6f-4dd7-a652-763268875bba admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:22.618667 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95252ea-cd6f-4dd7-a652-763268875bba admin admin] PATH_INFO: `/v3/projects/7fb15dd92fd0428caf31b9e9064a0c5b` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:22.619915 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95252ea-cd6f-4dd7-a652-763268875bba admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fb15dd92fd0428caf31b9e9064a0c5b)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:22.622083 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95252ea-cd6f-4dd7-a652-763268875bba admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:22.698025 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb1571-871a-4946-bb53-2cae5d544a4c service neutron] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:22.699421 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-03fb1571-871a-4946-bb53-2cae5d544a4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:22.710781 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb1571-871a-4946-bb53-2cae5d544a4c service neutron] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:22.714856 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4083841-5349-47d9-857e-5cb5ad44411e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:22.719967 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4083841-5349-47d9-857e-5cb5ad44411e service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:22.719967 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4083841-5349-47d9-857e-5cb5ad44411e service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:22.719967 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4083841-5349-47d9-857e-5cb5ad44411e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:22.719967 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-e4083841-5349-47d9-857e-5cb5ad44411e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:22.750277 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-d95252ea-cd6f-4dd7-a652-763268875bba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fb15dd92fd0428caf31b9e9064a0c5b', 'request_id': 'req-d95252ea-cd6f-4dd7-a652-763268875bba'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:22.756565 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-d95252ea-cd6f-4dd7-a652-763268875bba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fb15dd92fd0428caf31b9e9064a0c5b', 'request_id': 'req-d95252ea-cd6f-4dd7-a652-763268875bba'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:22.784752 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4083841-5349-47d9-857e-5cb5ad44411e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:22.786251 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4083841-5349-47d9-857e-5cb5ad44411e service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:22.796415 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1347/2702] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:02:22 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 15:02:22.834559 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-d95252ea-cd6f-4dd7-a652-763268875bba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fb15dd92fd0428caf31b9e9064a0c5b', 'request_id': 'req-d95252ea-cd6f-4dd7-a652-763268875bba'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:22.846704 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-d95252ea-cd6f-4dd7-a652-763268875bba admin admin] The token cache is being invalidate because project 7fb15dd92fd0428caf31b9e9064a0c5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:02:22.846996 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-d95252ea-cd6f-4dd7-a652-763268875bba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:22.847651 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-d95252ea-cd6f-4dd7-a652-763268875bba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:22.849427 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1356/2703] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 15:02:22 2022] DELETE /identity/v3/projects/7fb15dd92fd0428caf31b9e9064a0c5b => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:02:22.855492 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b0a103-0ba2-4625-98d8-96d49109ca5d None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:22.855877 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b0a103-0ba2-4625-98d8-96d49109ca5d None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:22.856259 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b0a103-0ba2-4625-98d8-96d49109ca5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:22.874734 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-81b0a103-0ba2-4625-98d8-96d49109ca5d None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:22.876066 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-81b0a103-0ba2-4625-98d8-96d49109ca5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:22.889040 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:22.936285 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1348/2704] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:02:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:02:22.944054 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b9195e-738f-4743-8079-66cd9299a3bc None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:22.944391 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b9195e-738f-4743-8079-66cd9299a3bc None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:22.944656 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b9195e-738f-4743-8079-66cd9299a3bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:22.964637 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-60b9195e-738f-4743-8079-66cd9299a3bc None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:22.968040 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-60b9195e-738f-4743-8079-66cd9299a3bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:22.974266 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1357/2705] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:02:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:02:22.978833 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b0a103-0ba2-4625-98d8-96d49109ca5d None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:22.989907 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b0a103-0ba2-4625-98d8-96d49109ca5d None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:22.996125 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580164aa-9217-4007-9cd4-4d0f4dd094f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:22.996818 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580164aa-9217-4007-9cd4-4d0f4dd094f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:22.997134 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580164aa-9217-4007-9cd4-4d0f4dd094f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:22.997437 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580164aa-9217-4007-9cd4-4d0f4dd094f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:22.998185 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580164aa-9217-4007-9cd4-4d0f4dd094f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:02:22.998488 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580164aa-9217-4007-9cd4-4d0f4dd094f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:23.000478 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580164aa-9217-4007-9cd4-4d0f4dd094f3 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:23.015732 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1349/2706] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jun 15 15:02:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:02:23.017586 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b9195e-738f-4743-8079-66cd9299a3bc None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:23.027391 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b9195e-738f-4743-8079-66cd9299a3bc None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:23.032353 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0c6d97-7099-43e9-9a31-ca3aeb764438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:23.033010 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c6d97-7099-43e9-9a31-ca3aeb764438 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:23.033296 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c6d97-7099-43e9-9a31-ca3aeb764438 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:23.033579 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c6d97-7099-43e9-9a31-ca3aeb764438 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:23.034280 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0c6d97-7099-43e9-9a31-ca3aeb764438 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:23.036195 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0c6d97-7099-43e9-9a31-ca3aeb764438 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:23.127665 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1358/2707] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 15:02:23 2022] POST /identity/v3/projects => generated 375 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:02:23.131912 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580164aa-9217-4007-9cd4-4d0f4dd094f3 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:23.142651 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580164aa-9217-4007-9cd4-4d0f4dd094f3 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:23.206868 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebe03bb-3d9b-4e42-9289-893558f429ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:23.207546 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebe03bb-3d9b-4e42-9289-893558f429ae admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:23.207835 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebe03bb-3d9b-4e42-9289-893558f429ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:23.208323 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebe03bb-3d9b-4e42-9289-893558f429ae admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:23.209173 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebe03bb-3d9b-4e42-9289-893558f429ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:23.211256 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebe03bb-3d9b-4e42-9289-893558f429ae admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:23.335192 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1350/2708] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 15:02:23 2022] POST /identity/v3/users => generated 288 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:02:23.343307 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0c6d97-7099-43e9-9a31-ca3aeb764438 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:23.347485 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0c6d97-7099-43e9-9a31-ca3aeb764438 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:23.353863 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a612e4-fc56-463e-8474-c7e322086bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:23.354561 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a612e4-fc56-463e-8474-c7e322086bc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:23.354843 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a612e4-fc56-463e-8474-c7e322086bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:23.355119 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a612e4-fc56-463e-8474-c7e322086bc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:23.355859 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a612e4-fc56-463e-8474-c7e322086bc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:23.357495 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a612e4-fc56-463e-8474-c7e322086bc3 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:23.372880 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1359/2709] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 15:02:23 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:02:23.377279 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebe03bb-3d9b-4e42-9289-893558f429ae admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:23.388040 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebe03bb-3d9b-4e42-9289-893558f429ae admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:23.393145 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95eca9e-6688-4eb0-a1b0-8554e2418719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:23.393896 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95eca9e-6688-4eb0-a1b0-8554e2418719 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:23.394248 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95eca9e-6688-4eb0-a1b0-8554e2418719 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:23.394616 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95eca9e-6688-4eb0-a1b0-8554e2418719 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:23.395414 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95eca9e-6688-4eb0-a1b0-8554e2418719 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:23.397143 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95eca9e-6688-4eb0-a1b0-8554e2418719 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:23.411598 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1351/2710] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 15:02:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 15:02:23.415160 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a612e4-fc56-463e-8474-c7e322086bc3 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:23.426115 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a612e4-fc56-463e-8474-c7e322086bc3 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:23.432854 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09033b34-8980-4737-a33c-ca5cc0f73512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:23.433741 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09033b34-8980-4737-a33c-ca5cc0f73512 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:23.434025 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09033b34-8980-4737-a33c-ca5cc0f73512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:23.434295 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09033b34-8980-4737-a33c-ca5cc0f73512 admin admin] PATH_INFO: `/v3/projects/05f7425f03bb48f492d4e24d4610355e/users/a4c6e34be183484baf403370289a1259/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:23.450682 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09033b34-8980-4737-a33c-ca5cc0f73512 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05f7425f03bb48f492d4e24d4610355e, user_id=a4c6e34be183484baf403370289a1259, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:23.453387 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09033b34-8980-4737-a33c-ca5cc0f73512 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:23.520789 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1360/2711] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 15:02:23 2022] PUT /identity/v3/projects/05f7425f03bb48f492d4e24d4610355e/users/a4c6e34be183484baf403370289a1259/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:02:23.531234 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a68c4b-f517-4641-8242-3bb47b1f50ef None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:23.531607 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a68c4b-f517-4641-8242-3bb47b1f50ef None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:23.531955 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a68c4b-f517-4641-8242-3bb47b1f50ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:23.563712 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-c0a68c4b-f517-4641-8242-3bb47b1f50ef None None] MFA Rules not processed for user `a4c6e34be183484baf403370289a1259`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:23.564952 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-c0a68c4b-f517-4641-8242-3bb47b1f50ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:23.634297 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1352/2712] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:02:23 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:02:23.646237 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09033b34-8980-4737-a33c-ca5cc0f73512 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:23.648004 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-09033b34-8980-4737-a33c-ca5cc0f73512 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:23.724907 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09033b34-8980-4737-a33c-ca5cc0f73512 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:23.732074 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ffc0ab-b934-4ca9-91b1-c23c62a95765 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:23.732801 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ffc0ab-b934-4ca9-91b1-c23c62a95765 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:23.733088 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ffc0ab-b934-4ca9-91b1-c23c62a95765 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:23.733656 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ffc0ab-b934-4ca9-91b1-c23c62a95765 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:23.743501 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ffc0ab-b934-4ca9-91b1-c23c62a95765 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:23.745003 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ffc0ab-b934-4ca9-91b1-c23c62a95765 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:23.755941 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1361/2713] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:02:23 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:02:28.900308 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a68c4b-f517-4641-8242-3bb47b1f50ef None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:28.917463 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:28.965588 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a68c4b-f517-4641-8242-3bb47b1f50ef None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:28.970144 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316029e7-568a-4868-b739-265b4108b3b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:28.971100 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316029e7-568a-4868-b739-265b4108b3b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:28.971382 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316029e7-568a-4868-b739-265b4108b3b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:28.971658 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316029e7-568a-4868-b739-265b4108b3b3 admin admin] PATH_INFO: `/v3/users/a4c6e34be183484baf403370289a1259` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:28.972880 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316029e7-568a-4868-b739-265b4108b3b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4c6e34be183484baf403370289a1259)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:28.974910 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316029e7-568a-4868-b739-265b4108b3b3 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:29.116473 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-316029e7-568a-4868-b739-265b4108b3b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4c6e34be183484baf403370289a1259'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:29.117325 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-316029e7-568a-4868-b739-265b4108b3b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4c6e34be183484baf403370289a1259'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:29.124765 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-316029e7-568a-4868-b739-265b4108b3b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4c6e34be183484baf403370289a1259'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:29.125397 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-316029e7-568a-4868-b739-265b4108b3b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4c6e34be183484baf403370289a1259'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:29.198980 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-316029e7-568a-4868-b739-265b4108b3b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4c6e34be183484baf403370289a1259'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:29.217638 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1353/2714] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 15:02:28 2022] DELETE /identity/v3/users/a4c6e34be183484baf403370289a1259 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:02:29.222209 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e141bbb5-8d59-419f-8696-61f47bf8ea20 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:29.222711 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e141bbb5-8d59-419f-8696-61f47bf8ea20 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:29.222823 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e141bbb5-8d59-419f-8696-61f47bf8ea20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:29.240247 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-e141bbb5-8d59-419f-8696-61f47bf8ea20 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:29.241440 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e141bbb5-8d59-419f-8696-61f47bf8ea20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:29.246323 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:29.302689 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1362/2715] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:02:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:02:29.313143 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316029e7-568a-4868-b739-265b4108b3b3 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:29.314564 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-316029e7-568a-4868-b739-265b4108b3b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:29.372332 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316029e7-568a-4868-b739-265b4108b3b3 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:29.376212 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c647f07e-8ab2-4e48-8fd5-83148ab3de4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:29.376882 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c647f07e-8ab2-4e48-8fd5-83148ab3de4a service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:29.377162 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c647f07e-8ab2-4e48-8fd5-83148ab3de4a service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:29.377437 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c647f07e-8ab2-4e48-8fd5-83148ab3de4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:29.386841 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c647f07e-8ab2-4e48-8fd5-83148ab3de4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:29.388341 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c647f07e-8ab2-4e48-8fd5-83148ab3de4a service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:29.397767 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1354/2716] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:02:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:02:29.818870 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e141bbb5-8d59-419f-8696-61f47bf8ea20 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:29.820502 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e141bbb5-8d59-419f-8696-61f47bf8ea20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:29.833431 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e141bbb5-8d59-419f-8696-61f47bf8ea20 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:29.838969 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb040c-047f-4484-9427-f3f445412e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:29.840130 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb040c-047f-4484-9427-f3f445412e10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:29.840286 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb040c-047f-4484-9427-f3f445412e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:29.840573 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb040c-047f-4484-9427-f3f445412e10 admin admin] PATH_INFO: `/v3/projects/05f7425f03bb48f492d4e24d4610355e` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:29.841776 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eb040c-047f-4484-9427-f3f445412e10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05f7425f03bb48f492d4e24d4610355e)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:29.843680 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eb040c-047f-4484-9427-f3f445412e10 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:29.978212 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-49eb040c-047f-4484-9427-f3f445412e10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05f7425f03bb48f492d4e24d4610355e', 'request_id': 'req-49eb040c-047f-4484-9427-f3f445412e10'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:29.984517 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-49eb040c-047f-4484-9427-f3f445412e10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05f7425f03bb48f492d4e24d4610355e', 'request_id': 'req-49eb040c-047f-4484-9427-f3f445412e10'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:30.056870 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-49eb040c-047f-4484-9427-f3f445412e10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05f7425f03bb48f492d4e24d4610355e', 'request_id': 'req-49eb040c-047f-4484-9427-f3f445412e10'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:30.074111 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-49eb040c-047f-4484-9427-f3f445412e10 admin admin] The token cache is being invalidate because project 05f7425f03bb48f492d4e24d4610355e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:02:30.074111 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-49eb040c-047f-4484-9427-f3f445412e10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:30.074111 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-49eb040c-047f-4484-9427-f3f445412e10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:30.074111 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1363/2717] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 15:02:29 2022] DELETE /identity/v3/projects/05f7425f03bb48f492d4e24d4610355e => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:02:30.076252 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab8ee42-c344-42d5-b25b-69ef46166412 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:30.076739 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab8ee42-c344-42d5-b25b-69ef46166412 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:30.076841 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab8ee42-c344-42d5-b25b-69ef46166412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:30.095409 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-eab8ee42-c344-42d5-b25b-69ef46166412 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:30.096633 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-eab8ee42-c344-42d5-b25b-69ef46166412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:30.130998 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:30.158032 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1355/2718] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:02:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:02:30.167268 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7654994-af6d-49d6-965b-1d2ffad78a3f None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:30.167559 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7654994-af6d-49d6-965b-1d2ffad78a3f None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:30.167843 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7654994-af6d-49d6-965b-1d2ffad78a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:30.186949 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-a7654994-af6d-49d6-965b-1d2ffad78a3f None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:30.188189 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a7654994-af6d-49d6-965b-1d2ffad78a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:30.197108 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1364/2719] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:02:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:02:30.201410 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab8ee42-c344-42d5-b25b-69ef46166412 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:30.211250 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab8ee42-c344-42d5-b25b-69ef46166412 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:30.215914 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011695a0-3880-43e4-aff4-0bbff1b99abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:30.216810 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011695a0-3880-43e4-aff4-0bbff1b99abe admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:30.217297 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011695a0-3880-43e4-aff4-0bbff1b99abe admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:30.217945 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011695a0-3880-43e4-aff4-0bbff1b99abe admin admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:30.218720 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011695a0-3880-43e4-aff4-0bbff1b99abe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:02:30.219037 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011695a0-3880-43e4-aff4-0bbff1b99abe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:30.220679 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011695a0-3880-43e4-aff4-0bbff1b99abe admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:30.232416 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1356/2720] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jun 15 15:02:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:02:30.236789 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7654994-af6d-49d6-965b-1d2ffad78a3f None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:30.246883 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7654994-af6d-49d6-965b-1d2ffad78a3f None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:30.251899 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095f5277-4b76-4a74-9a21-d11dd49690cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:30.252609 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095f5277-4b76-4a74-9a21-d11dd49690cb admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:30.252895 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095f5277-4b76-4a74-9a21-d11dd49690cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:30.253178 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095f5277-4b76-4a74-9a21-d11dd49690cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:30.253880 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095f5277-4b76-4a74-9a21-d11dd49690cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:30.255769 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095f5277-4b76-4a74-9a21-d11dd49690cb admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:30.357274 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1365/2721] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 15:02:30 2022] POST /identity/v3/projects => generated 371 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:02:30.361424 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011695a0-3880-43e4-aff4-0bbff1b99abe admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:30.372221 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011695a0-3880-43e4-aff4-0bbff1b99abe admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:30.433047 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5426d809-333f-491b-9588-7db804de1ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:30.433715 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5426d809-333f-491b-9588-7db804de1ecf admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:30.434002 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5426d809-333f-491b-9588-7db804de1ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:30.434281 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5426d809-333f-491b-9588-7db804de1ecf admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:30.434978 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5426d809-333f-491b-9588-7db804de1ecf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:30.436942 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5426d809-333f-491b-9588-7db804de1ecf admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:30.531510 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1357/2722] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 15:02:30 2022] POST /identity/v3/users => generated 286 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:02:30.535722 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095f5277-4b76-4a74-9a21-d11dd49690cb admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:30.546965 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095f5277-4b76-4a74-9a21-d11dd49690cb admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:30.552062 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3077b77-cbf0-49c4-a9a2-70d167cbbef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:30.552891 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3077b77-cbf0-49c4-a9a2-70d167cbbef0 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:30.553203 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3077b77-cbf0-49c4-a9a2-70d167cbbef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:30.553483 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3077b77-cbf0-49c4-a9a2-70d167cbbef0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:30.554220 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3077b77-cbf0-49c4-a9a2-70d167cbbef0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:30.556826 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3077b77-cbf0-49c4-a9a2-70d167cbbef0 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:30.568505 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1366/2723] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 15:02:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:02:30.572754 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5426d809-333f-491b-9588-7db804de1ecf admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:30.582848 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5426d809-333f-491b-9588-7db804de1ecf admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:30.587706 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92537de7-e615-4767-a5cf-20cec95bc7ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:30.588389 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92537de7-e615-4767-a5cf-20cec95bc7ee admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:30.588671 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92537de7-e615-4767-a5cf-20cec95bc7ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:30.588944 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92537de7-e615-4767-a5cf-20cec95bc7ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:30.589814 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92537de7-e615-4767-a5cf-20cec95bc7ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:30.591389 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92537de7-e615-4767-a5cf-20cec95bc7ee admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:30.602221 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1358/2724] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 15:02:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:02:30.606835 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3077b77-cbf0-49c4-a9a2-70d167cbbef0 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:30.622963 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3077b77-cbf0-49c4-a9a2-70d167cbbef0 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:30.622963 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31595699-802a-4ac9-a127-2a4b85b9c0fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:30.623702 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31595699-802a-4ac9-a127-2a4b85b9c0fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:30.624269 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31595699-802a-4ac9-a127-2a4b85b9c0fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:30.624546 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31595699-802a-4ac9-a127-2a4b85b9c0fd admin admin] PATH_INFO: `/v3/projects/726fdf7cb22a498fa47e93b95dd17300/users/4ed380579e9f4bfb99f8c3aca1e63444/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:30.639992 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31595699-802a-4ac9-a127-2a4b85b9c0fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=726fdf7cb22a498fa47e93b95dd17300, user_id=4ed380579e9f4bfb99f8c3aca1e63444, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:30.642713 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31595699-802a-4ac9-a127-2a4b85b9c0fd admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:30.708571 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1367/2725] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 15:02:30 2022] PUT /identity/v3/projects/726fdf7cb22a498fa47e93b95dd17300/users/4ed380579e9f4bfb99f8c3aca1e63444/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:02:30.719246 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf2b681-5c8c-44ac-a929-b2f0ad606c75 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:30.719537 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf2b681-5c8c-44ac-a929-b2f0ad606c75 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:30.719820 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf2b681-5c8c-44ac-a929-b2f0ad606c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:30.747084 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-caf2b681-5c8c-44ac-a929-b2f0ad606c75 None None] MFA Rules not processed for user `4ed380579e9f4bfb99f8c3aca1e63444`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:30.748320 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-caf2b681-5c8c-44ac-a929-b2f0ad606c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:30.811044 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1359/2726] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:02:30 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:02:30.823302 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31595699-802a-4ac9-a127-2a4b85b9c0fd admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:30.825048 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-31595699-802a-4ac9-a127-2a4b85b9c0fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:30.858921 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:30.884798 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31595699-802a-4ac9-a127-2a4b85b9c0fd admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:30.888930 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec038d2-4b95-41ee-90d4-4c83ffeab0b3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f18c57f27784694935bfb0bcea48a78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:30.889567 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec038d2-4b95-41ee-90d4-4c83ffeab0b3 service glance] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:30.889849 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec038d2-4b95-41ee-90d4-4c83ffeab0b3 service glance] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:30.890122 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec038d2-4b95-41ee-90d4-4c83ffeab0b3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:30.898972 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec038d2-4b95-41ee-90d4-4c83ffeab0b3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:30.900433 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec038d2-4b95-41ee-90d4-4c83ffeab0b3 service glance] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:30.910051 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1368/2727] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:02:30 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:02:40.808185 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=82,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:40.930951 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=13 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:03:05.700973 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf2b681-5c8c-44ac-a929-b2f0ad606c75 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:05.702398 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-caf2b681-5c8c-44ac-a929-b2f0ad606c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:05.779543 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf2b681-5c8c-44ac-a929-b2f0ad606c75 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:05.783507 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5780b1c-0b45-4274-b633-57533247fa46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:05.784178 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5780b1c-0b45-4274-b633-57533247fa46 service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:05.784462 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5780b1c-0b45-4274-b633-57533247fa46 service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:05.784739 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5780b1c-0b45-4274-b633-57533247fa46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:05.786527 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-c5780b1c-0b45-4274-b633-57533247fa46 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:05.867228 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5780b1c-0b45-4274-b633-57533247fa46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:05.868606 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5780b1c-0b45-4274-b633-57533247fa46 service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:05.887084 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1360/2728] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:03:05 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:03:11.680173 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec038d2-4b95-41ee-90d4-4c83ffeab0b3 service glance] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:11.744598 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec038d2-4b95-41ee-90d4-4c83ffeab0b3 service glance] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:11.750110 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47c1b27-7ee8-4628-b5f0-a7f7c25f4803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:11.750982 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47c1b27-7ee8-4628-b5f0-a7f7c25f4803 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:11.751261 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47c1b27-7ee8-4628-b5f0-a7f7c25f4803 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:11.751538 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47c1b27-7ee8-4628-b5f0-a7f7c25f4803 admin admin] PATH_INFO: `/v3/users/4ed380579e9f4bfb99f8c3aca1e63444` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:11.752718 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47c1b27-7ee8-4628-b5f0-a7f7c25f4803 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ed380579e9f4bfb99f8c3aca1e63444)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:11.754603 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47c1b27-7ee8-4628-b5f0-a7f7c25f4803 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:12.003724 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c47c1b27-7ee8-4628-b5f0-a7f7c25f4803 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ed380579e9f4bfb99f8c3aca1e63444'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:12.004830 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c47c1b27-7ee8-4628-b5f0-a7f7c25f4803 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ed380579e9f4bfb99f8c3aca1e63444'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:12.006270 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c47c1b27-7ee8-4628-b5f0-a7f7c25f4803 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ed380579e9f4bfb99f8c3aca1e63444'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:12.090502 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c47c1b27-7ee8-4628-b5f0-a7f7c25f4803 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ed380579e9f4bfb99f8c3aca1e63444'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:12.103261 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c47c1b27-7ee8-4628-b5f0-a7f7c25f4803 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ed380579e9f4bfb99f8c3aca1e63444'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:12.116401 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1369/2729] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 15:03:11 2022] DELETE /identity/v3/users/4ed380579e9f4bfb99f8c3aca1e63444 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:03:12.121647 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd569c-e8da-4d89-b65b-fcbcccefd944 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:12.122202 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd569c-e8da-4d89-b65b-fcbcccefd944 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:12.122202 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd569c-e8da-4d89-b65b-fcbcccefd944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:12.139267 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-9efd569c-e8da-4d89-b65b-fcbcccefd944 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:03:12.140982 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-9efd569c-e8da-4d89-b65b-fcbcccefd944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:12.202435 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1361/2730] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:03:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:03:12.212825 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47c1b27-7ee8-4628-b5f0-a7f7c25f4803 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:12.214548 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c47c1b27-7ee8-4628-b5f0-a7f7c25f4803 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:12.283332 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47c1b27-7ee8-4628-b5f0-a7f7c25f4803 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:12.286980 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab9440c-6106-4a9c-b589-410b0fd71bd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:12.287622 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab9440c-6106-4a9c-b589-410b0fd71bd5 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:12.287900 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab9440c-6106-4a9c-b589-410b0fd71bd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:12.288203 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab9440c-6106-4a9c-b589-410b0fd71bd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:12.298274 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab9440c-6106-4a9c-b589-410b0fd71bd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:12.299699 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab9440c-6106-4a9c-b589-410b0fd71bd5 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:12.310229 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1370/2731] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:03:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:03:12.663680 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efd569c-e8da-4d89-b65b-fcbcccefd944 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:12.665269 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-9efd569c-e8da-4d89-b65b-fcbcccefd944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:12.681806 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efd569c-e8da-4d89-b65b-fcbcccefd944 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:12.687158 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb54d6c-7600-4ff8-8995-b46ff12b9cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:12.687940 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb54d6c-7600-4ff8-8995-b46ff12b9cc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:12.688629 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb54d6c-7600-4ff8-8995-b46ff12b9cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:12.688916 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb54d6c-7600-4ff8-8995-b46ff12b9cc3 admin admin] PATH_INFO: `/v3/projects/726fdf7cb22a498fa47e93b95dd17300` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:12.690213 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb54d6c-7600-4ff8-8995-b46ff12b9cc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=726fdf7cb22a498fa47e93b95dd17300)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:12.692202 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb54d6c-7600-4ff8-8995-b46ff12b9cc3 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:12.693956 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:03:12.818170 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-8fb54d6c-7600-4ff8-8995-b46ff12b9cc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '726fdf7cb22a498fa47e93b95dd17300', 'request_id': 'req-8fb54d6c-7600-4ff8-8995-b46ff12b9cc3'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:12.873027 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-8fb54d6c-7600-4ff8-8995-b46ff12b9cc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '726fdf7cb22a498fa47e93b95dd17300', 'request_id': 'req-8fb54d6c-7600-4ff8-8995-b46ff12b9cc3'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:12.879043 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-8fb54d6c-7600-4ff8-8995-b46ff12b9cc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '726fdf7cb22a498fa47e93b95dd17300', 'request_id': 'req-8fb54d6c-7600-4ff8-8995-b46ff12b9cc3'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:12.890772 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-8fb54d6c-7600-4ff8-8995-b46ff12b9cc3 admin admin] The token cache is being invalidate because project 726fdf7cb22a498fa47e93b95dd17300 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:03:12.891055 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-8fb54d6c-7600-4ff8-8995-b46ff12b9cc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:12.891686 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-8fb54d6c-7600-4ff8-8995-b46ff12b9cc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:12.893247 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1362/2732] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 15:03:12 2022] DELETE /identity/v3/projects/726fdf7cb22a498fa47e93b95dd17300 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:03:12.898218 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d324f6-5ed0-4430-86ee-92764a2f23f8 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:12.898683 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d324f6-5ed0-4430-86ee-92764a2f23f8 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:12.898782 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d324f6-5ed0-4430-86ee-92764a2f23f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:12.917371 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-64d324f6-5ed0-4430-86ee-92764a2f23f8 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:03:12.918675 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-64d324f6-5ed0-4430-86ee-92764a2f23f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:12.935400 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:03:12.978311 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1371/2733] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:03:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:03:12.987411 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d518cd0e-29e0-46ea-8c5f-7b1c641c75ed None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:12.987699 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d518cd0e-29e0-46ea-8c5f-7b1c641c75ed None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:12.987979 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d518cd0e-29e0-46ea-8c5f-7b1c641c75ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:13.005969 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-d518cd0e-29e0-46ea-8c5f-7b1c641c75ed None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:03:13.007153 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-d518cd0e-29e0-46ea-8c5f-7b1c641c75ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:13.015385 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1363/2734] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:03:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:03:13.019466 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d324f6-5ed0-4430-86ee-92764a2f23f8 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:13.029831 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d324f6-5ed0-4430-86ee-92764a2f23f8 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:13.034417 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57f0fc3-eb56-437e-8074-1457ea2b7642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:13.035066 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57f0fc3-eb56-437e-8074-1457ea2b7642 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:13.035351 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57f0fc3-eb56-437e-8074-1457ea2b7642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:13.035628 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57f0fc3-eb56-437e-8074-1457ea2b7642 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:13.036389 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57f0fc3-eb56-437e-8074-1457ea2b7642 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:03:13.036700 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57f0fc3-eb56-437e-8074-1457ea2b7642 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:13.038278 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57f0fc3-eb56-437e-8074-1457ea2b7642 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:13.049888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1372/2735] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jun 15 15:03:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:03:13.053920 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d518cd0e-29e0-46ea-8c5f-7b1c641c75ed None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:13.063005 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d518cd0e-29e0-46ea-8c5f-7b1c641c75ed None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:13.067566 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b1f44e-d787-42c1-afe4-de2cee88dd0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:13.068294 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b1f44e-d787-42c1-afe4-de2cee88dd0e admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:13.068529 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b1f44e-d787-42c1-afe4-de2cee88dd0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:13.068807 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b1f44e-d787-42c1-afe4-de2cee88dd0e admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:13.069501 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b1f44e-d787-42c1-afe4-de2cee88dd0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:13.071387 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b1f44e-d787-42c1-afe4-de2cee88dd0e admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:13.142716 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1364/2736] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 15:03:13 2022] POST /identity/v3/projects => generated 365 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:03:13.146887 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57f0fc3-eb56-437e-8074-1457ea2b7642 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:13.158220 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57f0fc3-eb56-437e-8074-1457ea2b7642 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:13.217278 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78446c9-e5e0-4ae9-bba5-2aef6fc1e354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:13.217941 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78446c9-e5e0-4ae9-bba5-2aef6fc1e354 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:13.218222 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78446c9-e5e0-4ae9-bba5-2aef6fc1e354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:13.218500 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78446c9-e5e0-4ae9-bba5-2aef6fc1e354 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:13.219192 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78446c9-e5e0-4ae9-bba5-2aef6fc1e354 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:13.221137 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78446c9-e5e0-4ae9-bba5-2aef6fc1e354 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:13.319461 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1373/2737] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 15:03:13 2022] POST /identity/v3/users => generated 283 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:03:13.327802 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b1f44e-d787-42c1-afe4-de2cee88dd0e admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:13.337156 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b1f44e-d787-42c1-afe4-de2cee88dd0e admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:13.342198 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e4994b-6a87-4dc0-a117-1eb14beaa076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:13.342920 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e4994b-6a87-4dc0-a117-1eb14beaa076 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:13.343161 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e4994b-6a87-4dc0-a117-1eb14beaa076 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:13.343517 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e4994b-6a87-4dc0-a117-1eb14beaa076 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:13.344188 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e4994b-6a87-4dc0-a117-1eb14beaa076 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:13.345770 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e4994b-6a87-4dc0-a117-1eb14beaa076 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:13.356891 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1365/2738] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 15:03:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:03:13.363672 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78446c9-e5e0-4ae9-bba5-2aef6fc1e354 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:13.373131 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78446c9-e5e0-4ae9-bba5-2aef6fc1e354 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:13.379578 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d6a038-a634-4963-a82b-614b99a236ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:13.380293 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d6a038-a634-4963-a82b-614b99a236ec admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:13.380583 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d6a038-a634-4963-a82b-614b99a236ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:13.380863 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d6a038-a634-4963-a82b-614b99a236ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:13.381600 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d6a038-a634-4963-a82b-614b99a236ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:13.383204 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d6a038-a634-4963-a82b-614b99a236ec admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:13.398351 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1374/2739] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 15:03:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:03:13.399615 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e4994b-6a87-4dc0-a117-1eb14beaa076 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:13.410497 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e4994b-6a87-4dc0-a117-1eb14beaa076 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:13.415407 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3d8530-84d3-4c4a-a6ad-834fd742ebb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:13.416368 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d8530-84d3-4c4a-a6ad-834fd742ebb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:13.416652 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d8530-84d3-4c4a-a6ad-834fd742ebb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:13.416936 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d8530-84d3-4c4a-a6ad-834fd742ebb8 admin admin] PATH_INFO: `/v3/projects/d8e25187a9d5455bad89ba02bc15ea18/users/e2bcdc3bd4b04083aa7c8ebb3215c166/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:13.431735 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3d8530-84d3-4c4a-a6ad-834fd742ebb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8e25187a9d5455bad89ba02bc15ea18, user_id=e2bcdc3bd4b04083aa7c8ebb3215c166, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:13.435111 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3d8530-84d3-4c4a-a6ad-834fd742ebb8 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:13.629330 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1366/2740] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 15:03:13 2022] PUT /identity/v3/projects/d8e25187a9d5455bad89ba02bc15ea18/users/e2bcdc3bd4b04083aa7c8ebb3215c166/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:03:13.639243 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e783159-05e9-47f9-ba6b-fd44591ad301 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:13.639535 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e783159-05e9-47f9-ba6b-fd44591ad301 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:13.639818 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e783159-05e9-47f9-ba6b-fd44591ad301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:13.669119 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-8e783159-05e9-47f9-ba6b-fd44591ad301 None None] MFA Rules not processed for user `e2bcdc3bd4b04083aa7c8ebb3215c166`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:03:13.670339 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-8e783159-05e9-47f9-ba6b-fd44591ad301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:13.736725 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1375/2741] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:03:13 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:03:13.749342 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3d8530-84d3-4c4a-a6ad-834fd742ebb8 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:13.753164 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-4c3d8530-84d3-4c4a-a6ad-834fd742ebb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:13.810415 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3d8530-84d3-4c4a-a6ad-834fd742ebb8 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:13.814166 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a190dc-8fca-4278-ab53-78cc07895d9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:13.814803 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a190dc-8fca-4278-ab53-78cc07895d9c service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:13.815086 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a190dc-8fca-4278-ab53-78cc07895d9c service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:13.815359 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a190dc-8fca-4278-ab53-78cc07895d9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:13.823532 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a190dc-8fca-4278-ab53-78cc07895d9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:13.824974 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a190dc-8fca-4278-ab53-78cc07895d9c service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:13.833597 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1367/2742] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:03:13 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:03:14.700257 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e783159-05e9-47f9-ba6b-fd44591ad301 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:14.702510 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-8e783159-05e9-47f9-ba6b-fd44591ad301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:14.729108 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:03:14.811232 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e783159-05e9-47f9-ba6b-fd44591ad301 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:14.811232 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7f8840-c4b8-4603-ad39-f46a6d033844 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bb6671c19464424a6e9cd4519ac93e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:14.811232 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f8840-c4b8-4603-ad39-f46a6d033844 service placement] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:14.811232 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f8840-c4b8-4603-ad39-f46a6d033844 service placement] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:14.813832 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f8840-c4b8-4603-ad39-f46a6d033844 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:14.814247 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a190dc-8fca-4278-ab53-78cc07895d9c service cinder] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:14.814622 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-de7f8840-c4b8-4603-ad39-f46a6d033844 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:14.814824 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-e4a190dc-8fca-4278-ab53-78cc07895d9c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:14.827131 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7f8840-c4b8-4603-ad39-f46a6d033844 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:14.829231 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7f8840-c4b8-4603-ad39-f46a6d033844 service placement] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:14.840476 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1376/2743] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:03:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:03:14.847905 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:03:14.874578 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a190dc-8fca-4278-ab53-78cc07895d9c service cinder] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:14.878200 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c36b796-57d5-42b7-91d3-1c52220b3b42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:14.878853 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c36b796-57d5-42b7-91d3-1c52220b3b42 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:14.879192 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c36b796-57d5-42b7-91d3-1c52220b3b42 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:14.879472 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c36b796-57d5-42b7-91d3-1c52220b3b42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:14.881308 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-5c36b796-57d5-42b7-91d3-1c52220b3b42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:14.944402 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c36b796-57d5-42b7-91d3-1c52220b3b42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:14.945866 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c36b796-57d5-42b7-91d3-1c52220b3b42 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:14.956898 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1368/2744] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:03:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:03:18.647597 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7f8840-c4b8-4603-ad39-f46a6d033844 service placement] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:18.649119 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-de7f8840-c4b8-4603-ad39-f46a6d033844 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:18.663044 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7f8840-c4b8-4603-ad39-f46a6d033844 service placement] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:18.667618 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da6ed4d-8b58-4986-ba78-9408cfe3a44b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:18.668513 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da6ed4d-8b58-4986-ba78-9408cfe3a44b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:18.668801 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da6ed4d-8b58-4986-ba78-9408cfe3a44b admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:18.669080 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da6ed4d-8b58-4986-ba78-9408cfe3a44b admin admin] PATH_INFO: `/v3/users/2dcc02eb960e45b99587438aa3696f40` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:18.670486 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da6ed4d-8b58-4986-ba78-9408cfe3a44b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2dcc02eb960e45b99587438aa3696f40)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:18.672417 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da6ed4d-8b58-4986-ba78-9408cfe3a44b admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:18.846160 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-2da6ed4d-8b58-4986-ba78-9408cfe3a44b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2dcc02eb960e45b99587438aa3696f40'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:18.846813 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-2da6ed4d-8b58-4986-ba78-9408cfe3a44b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2dcc02eb960e45b99587438aa3696f40'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:18.847407 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-2da6ed4d-8b58-4986-ba78-9408cfe3a44b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2dcc02eb960e45b99587438aa3696f40'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:18.913118 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-2da6ed4d-8b58-4986-ba78-9408cfe3a44b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2dcc02eb960e45b99587438aa3696f40'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:18.925767 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-2da6ed4d-8b58-4986-ba78-9408cfe3a44b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2dcc02eb960e45b99587438aa3696f40'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:18.938220 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1377/2745] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 15:03:18 2022] DELETE /identity/v3/users/2dcc02eb960e45b99587438aa3696f40 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:03:19.160461 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c36b796-57d5-42b7-91d3-1c52220b3b42 service neutron] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:19.172580 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c36b796-57d5-42b7-91d3-1c52220b3b42 service neutron] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:19.229876 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe8e6cd-075a-49fb-bba4-1d7675e5ad38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:19.230732 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe8e6cd-075a-49fb-bba4-1d7675e5ad38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:19.231012 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe8e6cd-075a-49fb-bba4-1d7675e5ad38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:19.231286 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe8e6cd-075a-49fb-bba4-1d7675e5ad38 admin admin] PATH_INFO: `/v3/projects/f9c28220b6ea4db99f81a37e0794450b` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:19.232546 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe8e6cd-075a-49fb-bba4-1d7675e5ad38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9c28220b6ea4db99f81a37e0794450b)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:19.234422 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe8e6cd-075a-49fb-bba4-1d7675e5ad38 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:19.313370 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-1fe8e6cd-075a-49fb-bba4-1d7675e5ad38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9c28220b6ea4db99f81a37e0794450b', 'request_id': 'req-1fe8e6cd-075a-49fb-bba4-1d7675e5ad38'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:19.364637 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-1fe8e6cd-075a-49fb-bba4-1d7675e5ad38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9c28220b6ea4db99f81a37e0794450b', 'request_id': 'req-1fe8e6cd-075a-49fb-bba4-1d7675e5ad38'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:19.370341 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-1fe8e6cd-075a-49fb-bba4-1d7675e5ad38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9c28220b6ea4db99f81a37e0794450b', 'request_id': 'req-1fe8e6cd-075a-49fb-bba4-1d7675e5ad38'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:19.382129 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-1fe8e6cd-075a-49fb-bba4-1d7675e5ad38 admin admin] The token cache is being invalidate because project f9c28220b6ea4db99f81a37e0794450b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:03:19.382414 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-1fe8e6cd-075a-49fb-bba4-1d7675e5ad38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:19.383039 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-1fe8e6cd-075a-49fb-bba4-1d7675e5ad38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:19.384627 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1369/2746] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 15:03:19 2022] DELETE /identity/v3/projects/f9c28220b6ea4db99f81a37e0794450b => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:03:19.390289 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd6b489-efbc-40ff-8c4e-a540be24b330 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:19.390577 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd6b489-efbc-40ff-8c4e-a540be24b330 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:19.390864 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd6b489-efbc-40ff-8c4e-a540be24b330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:19.408975 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-4dd6b489-efbc-40ff-8c4e-a540be24b330 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:03:19.410179 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-4dd6b489-efbc-40ff-8c4e-a540be24b330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:19.465947 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:03:19.475170 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1378/2747] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:03:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:03:19.484035 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08daa263-8fcd-4b78-98cd-012e63a9f2d9 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:19.484721 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08daa263-8fcd-4b78-98cd-012e63a9f2d9 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:19.484721 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08daa263-8fcd-4b78-98cd-012e63a9f2d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:19.504403 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-08daa263-8fcd-4b78-98cd-012e63a9f2d9 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:03:19.505641 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-08daa263-8fcd-4b78-98cd-012e63a9f2d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:19.514045 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1370/2748] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:03:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:03:19.518402 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd6b489-efbc-40ff-8c4e-a540be24b330 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:19.529008 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd6b489-efbc-40ff-8c4e-a540be24b330 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:19.533658 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa760b5c-68be-4702-bd30-f52be6017e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:19.534312 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa760b5c-68be-4702-bd30-f52be6017e59 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:19.534599 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa760b5c-68be-4702-bd30-f52be6017e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:19.534878 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa760b5c-68be-4702-bd30-f52be6017e59 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:19.535672 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa760b5c-68be-4702-bd30-f52be6017e59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:03:19.535984 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa760b5c-68be-4702-bd30-f52be6017e59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:19.537705 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa760b5c-68be-4702-bd30-f52be6017e59 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:19.550223 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1379/2749] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jun 15 15:03:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:03:19.554702 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08daa263-8fcd-4b78-98cd-012e63a9f2d9 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:19.559442 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:03:19.566003 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08daa263-8fcd-4b78-98cd-012e63a9f2d9 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:19.571071 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c15c54-5c56-4f9b-b3c7-6c48805ea668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:19.571731 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c15c54-5c56-4f9b-b3c7-6c48805ea668 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:19.572018 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c15c54-5c56-4f9b-b3c7-6c48805ea668 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:19.572327 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c15c54-5c56-4f9b-b3c7-6c48805ea668 admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:19.573019 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c15c54-5c56-4f9b-b3c7-6c48805ea668 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:19.574933 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c15c54-5c56-4f9b-b3c7-6c48805ea668 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:19.658792 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1371/2750] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 15:03:19 2022] POST /identity/v3/projects => generated 383 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:03:19.662917 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa760b5c-68be-4702-bd30-f52be6017e59 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:19.673545 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa760b5c-68be-4702-bd30-f52be6017e59 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:19.730115 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4f2fdb-b80a-4492-94b7-a4d99425b783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:19.730769 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4f2fdb-b80a-4492-94b7-a4d99425b783 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:19.731055 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4f2fdb-b80a-4492-94b7-a4d99425b783 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:19.731334 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4f2fdb-b80a-4492-94b7-a4d99425b783 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:19.732805 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4f2fdb-b80a-4492-94b7-a4d99425b783 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:19.734728 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4f2fdb-b80a-4492-94b7-a4d99425b783 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:19.823737 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1380/2751] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 15:03:19 2022] POST /identity/v3/users => generated 292 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:03:19.827938 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c15c54-5c56-4f9b-b3c7-6c48805ea668 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:19.838191 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c15c54-5c56-4f9b-b3c7-6c48805ea668 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:19.843355 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd909f2-3435-409e-828c-4082c8732604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:19.845016 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd909f2-3435-409e-828c-4082c8732604 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:19.845016 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd909f2-3435-409e-828c-4082c8732604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:19.845016 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd909f2-3435-409e-828c-4082c8732604 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:19.845371 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd909f2-3435-409e-828c-4082c8732604 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:19.846975 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd909f2-3435-409e-828c-4082c8732604 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:19.857771 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1372/2752] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 15:03:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:03:19.861773 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4f2fdb-b80a-4492-94b7-a4d99425b783 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:19.871996 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4f2fdb-b80a-4492-94b7-a4d99425b783 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:19.877050 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61fa189-1c15-43e5-aa3c-7b4cd598bc35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:19.877711 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61fa189-1c15-43e5-aa3c-7b4cd598bc35 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:19.877997 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61fa189-1c15-43e5-aa3c-7b4cd598bc35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:19.878276 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61fa189-1c15-43e5-aa3c-7b4cd598bc35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:19.879000 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61fa189-1c15-43e5-aa3c-7b4cd598bc35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:19.880634 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61fa189-1c15-43e5-aa3c-7b4cd598bc35 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:19.891271 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1381/2753] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 15:03:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:03:19.895412 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd909f2-3435-409e-828c-4082c8732604 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:19.905979 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd909f2-3435-409e-828c-4082c8732604 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:19.912040 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0708a56d-5c75-4a8e-a76f-701cfd6da131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:19.912940 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0708a56d-5c75-4a8e-a76f-701cfd6da131 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:19.913218 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0708a56d-5c75-4a8e-a76f-701cfd6da131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:19.913490 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0708a56d-5c75-4a8e-a76f-701cfd6da131 admin admin] PATH_INFO: `/v3/projects/e6829672292b4494931500de38f4d924/users/17f8c496bb0049719cd67c457ba40f35/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:19.928619 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0708a56d-5c75-4a8e-a76f-701cfd6da131 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6829672292b4494931500de38f4d924, user_id=17f8c496bb0049719cd67c457ba40f35, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:19.931168 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0708a56d-5c75-4a8e-a76f-701cfd6da131 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:20.005671 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1373/2754] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 15:03:19 2022] PUT /identity/v3/projects/e6829672292b4494931500de38f4d924/users/17f8c496bb0049719cd67c457ba40f35/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:03:20.010773 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762b0ec9-3ded-487d-903f-136bef01427c None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:20.011176 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762b0ec9-3ded-487d-903f-136bef01427c None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:20.011529 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762b0ec9-3ded-487d-903f-136bef01427c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:20.030667 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-762b0ec9-3ded-487d-903f-136bef01427c None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:03:20.031970 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-762b0ec9-3ded-487d-903f-136bef01427c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:20.095833 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1382/2755] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:03:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:03:20.105465 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0708a56d-5c75-4a8e-a76f-701cfd6da131 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:20.107902 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-0708a56d-5c75-4a8e-a76f-701cfd6da131 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:20.164546 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0708a56d-5c75-4a8e-a76f-701cfd6da131 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:20.168241 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46958a69-4145-4d50-a348-9f965f718d44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:20.168878 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46958a69-4145-4d50-a348-9f965f718d44 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:20.169156 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46958a69-4145-4d50-a348-9f965f718d44 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:20.170162 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46958a69-4145-4d50-a348-9f965f718d44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:20.184157 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46958a69-4145-4d50-a348-9f965f718d44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:20.184157 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46958a69-4145-4d50-a348-9f965f718d44 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:20.191379 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1374/2756] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:03:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:03:25.313475 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40c2965-1f39-4374-a354-443bffa3eb66 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:25.314342 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40c2965-1f39-4374-a354-443bffa3eb66 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:25.314621 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40c2965-1f39-4374-a354-443bffa3eb66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:25.350254 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-d40c2965-1f39-4374-a354-443bffa3eb66 None None] MFA Rules not processed for user `17f8c496bb0049719cd67c457ba40f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:03:25.351490 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-d40c2965-1f39-4374-a354-443bffa3eb66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:25.416904 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:03:25.427290 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1383/2757] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:03:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:03:25.442416 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46958a69-4145-4d50-a348-9f965f718d44 service neutron] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:25.443670 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-46958a69-4145-4d50-a348-9f965f718d44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:25.581960 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46958a69-4145-4d50-a348-9f965f718d44 service neutron] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:25.589256 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1d6dc5-bb68-4132-9da8-d60121eb4c7e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0927c05c340c48e48b18754888aa9c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:25.590594 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1d6dc5-bb68-4132-9da8-d60121eb4c7e service nova] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:25.591101 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1d6dc5-bb68-4132-9da8-d60121eb4c7e service nova] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:25.591495 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1d6dc5-bb68-4132-9da8-d60121eb4c7e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:25.601736 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1d6dc5-bb68-4132-9da8-d60121eb4c7e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:25.603817 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1d6dc5-bb68-4132-9da8-d60121eb4c7e service nova] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:25.615246 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1375/2758] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:03:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:03:28.141445 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d493c447-eef8-4a43-8499-7a2337d7ec35 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:28.141979 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d493c447-eef8-4a43-8499-7a2337d7ec35 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:28.143725 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d493c447-eef8-4a43-8499-7a2337d7ec35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:28.179477 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-d493c447-eef8-4a43-8499-7a2337d7ec35 None None] MFA Rules not processed for user `17f8c496bb0049719cd67c457ba40f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:03:28.182027 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-d493c447-eef8-4a43-8499-7a2337d7ec35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:28.192364 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1384/2759] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:03:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:03:28.575409 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1d6dc5-bb68-4132-9da8-d60121eb4c7e service nova] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:28.585816 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1d6dc5-bb68-4132-9da8-d60121eb4c7e service nova] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:28.590521 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6a7ab9-6e55-4a1e-a7d3-fb190b954d3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:28.590774 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a7ab9-6e55-4a1e-a7d3-fb190b954d3e service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:28.590919 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a7ab9-6e55-4a1e-a7d3-fb190b954d3e service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:28.591203 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a7ab9-6e55-4a1e-a7d3-fb190b954d3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:28.593384 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-8f6a7ab9-6e55-4a1e-a7d3-fb190b954d3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:28.600239 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:03:28.604779 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6a7ab9-6e55-4a1e-a7d3-fb190b954d3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:28.606248 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6a7ab9-6e55-4a1e-a7d3-fb190b954d3e service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:28.616079 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1376/2760] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:03:28 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:03:38.167641 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=5 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:03:38.724325 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:04:07.169804 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d493c447-eef8-4a43-8499-7a2337d7ec35 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:07.171208 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-d493c447-eef8-4a43-8499-7a2337d7ec35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:07.185186 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d493c447-eef8-4a43-8499-7a2337d7ec35 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:07.189986 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8454c34d-4b82-420c-a721-e9c0e01a6ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:07.190858 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8454c34d-4b82-420c-a721-e9c0e01a6ba0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:07.191139 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8454c34d-4b82-420c-a721-e9c0e01a6ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:07.191417 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8454c34d-4b82-420c-a721-e9c0e01a6ba0 admin admin] PATH_INFO: `/v3/users/e2bcdc3bd4b04083aa7c8ebb3215c166` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:07.192626 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8454c34d-4b82-420c-a721-e9c0e01a6ba0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2bcdc3bd4b04083aa7c8ebb3215c166)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:07.194536 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8454c34d-4b82-420c-a721-e9c0e01a6ba0 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:07.393834 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8454c34d-4b82-420c-a721-e9c0e01a6ba0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2bcdc3bd4b04083aa7c8ebb3215c166'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:07.394529 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8454c34d-4b82-420c-a721-e9c0e01a6ba0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2bcdc3bd4b04083aa7c8ebb3215c166'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:07.395170 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8454c34d-4b82-420c-a721-e9c0e01a6ba0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2bcdc3bd4b04083aa7c8ebb3215c166'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:07.467387 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8454c34d-4b82-420c-a721-e9c0e01a6ba0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2bcdc3bd4b04083aa7c8ebb3215c166'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:07.481398 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8454c34d-4b82-420c-a721-e9c0e01a6ba0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2bcdc3bd4b04083aa7c8ebb3215c166'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:07.494129 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1385/2761] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 15:04:07 2022] DELETE /identity/v3/users/e2bcdc3bd4b04083aa7c8ebb3215c166 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:04:07.499029 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09aa457-3980-4a7e-bb6a-c0b7b1b703c2 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:07.499536 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09aa457-3980-4a7e-bb6a-c0b7b1b703c2 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:07.499648 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09aa457-3980-4a7e-bb6a-c0b7b1b703c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:07.518077 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-e09aa457-3980-4a7e-bb6a-c0b7b1b703c2 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:04:07.519266 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-e09aa457-3980-4a7e-bb6a-c0b7b1b703c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:07.577715 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1377/2762] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:04:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:04:07.596694 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8454c34d-4b82-420c-a721-e9c0e01a6ba0 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:07.598163 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-8454c34d-4b82-420c-a721-e9c0e01a6ba0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:07.656582 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8454c34d-4b82-420c-a721-e9c0e01a6ba0 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:07.660804 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5d21b6-78ca-4cfa-9847-b08f6eb2e559 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:07.661448 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5d21b6-78ca-4cfa-9847-b08f6eb2e559 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:07.661727 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5d21b6-78ca-4cfa-9847-b08f6eb2e559 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:07.662005 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5d21b6-78ca-4cfa-9847-b08f6eb2e559 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:07.671212 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5d21b6-78ca-4cfa-9847-b08f6eb2e559 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:07.672676 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5d21b6-78ca-4cfa-9847-b08f6eb2e559 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:07.682119 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1386/2763] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:04:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:04:08.110825 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09aa457-3980-4a7e-bb6a-c0b7b1b703c2 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:08.111042 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-e09aa457-3980-4a7e-bb6a-c0b7b1b703c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:08.127977 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09aa457-3980-4a7e-bb6a-c0b7b1b703c2 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:08.141099 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a4a5e5-636b-4952-a5dc-edf243cce44c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:08.142499 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a4a5e5-636b-4952-a5dc-edf243cce44c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:08.142796 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a4a5e5-636b-4952-a5dc-edf243cce44c admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:08.143082 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a4a5e5-636b-4952-a5dc-edf243cce44c admin admin] PATH_INFO: `/v3/projects/d8e25187a9d5455bad89ba02bc15ea18` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:08.144366 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a4a5e5-636b-4952-a5dc-edf243cce44c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8e25187a9d5455bad89ba02bc15ea18)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:08.146300 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a4a5e5-636b-4952-a5dc-edf243cce44c admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:08.252514 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-33a4a5e5-636b-4952-a5dc-edf243cce44c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8e25187a9d5455bad89ba02bc15ea18', 'request_id': 'req-33a4a5e5-636b-4952-a5dc-edf243cce44c'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:08.341271 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-33a4a5e5-636b-4952-a5dc-edf243cce44c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8e25187a9d5455bad89ba02bc15ea18', 'request_id': 'req-33a4a5e5-636b-4952-a5dc-edf243cce44c'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:08.347512 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-33a4a5e5-636b-4952-a5dc-edf243cce44c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8e25187a9d5455bad89ba02bc15ea18', 'request_id': 'req-33a4a5e5-636b-4952-a5dc-edf243cce44c'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:08.359238 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-33a4a5e5-636b-4952-a5dc-edf243cce44c admin admin] The token cache is being invalidate because project d8e25187a9d5455bad89ba02bc15ea18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:04:08.359513 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-33a4a5e5-636b-4952-a5dc-edf243cce44c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:08.360953 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-33a4a5e5-636b-4952-a5dc-edf243cce44c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:08.362519 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1378/2764] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 15:04:08 2022] DELETE /identity/v3/projects/d8e25187a9d5455bad89ba02bc15ea18 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:04:08.367615 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dfddc6-ee55-4fd4-a31c-b0d0c5fb2e5f None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:08.368125 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dfddc6-ee55-4fd4-a31c-b0d0c5fb2e5f None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:08.368245 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dfddc6-ee55-4fd4-a31c-b0d0c5fb2e5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:08.386941 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-78dfddc6-ee55-4fd4-a31c-b0d0c5fb2e5f None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:04:08.388224 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-78dfddc6-ee55-4fd4-a31c-b0d0c5fb2e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:08.449185 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1387/2765] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:04:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:04:08.457828 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0bb9f5-57ee-46b4-aac6-879b1289f026 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:08.458117 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0bb9f5-57ee-46b4-aac6-879b1289f026 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:08.458398 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0bb9f5-57ee-46b4-aac6-879b1289f026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:08.478421 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-fc0bb9f5-57ee-46b4-aac6-879b1289f026 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:04:08.479660 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-fc0bb9f5-57ee-46b4-aac6-879b1289f026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:08.488207 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1379/2766] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:04:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:04:08.492778 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dfddc6-ee55-4fd4-a31c-b0d0c5fb2e5f None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:08.503123 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dfddc6-ee55-4fd4-a31c-b0d0c5fb2e5f None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:08.507656 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b12c97-43de-4f41-bc6e-b1f5aa6bec45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:08.508338 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b12c97-43de-4f41-bc6e-b1f5aa6bec45 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:08.508619 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b12c97-43de-4f41-bc6e-b1f5aa6bec45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:08.508892 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b12c97-43de-4f41-bc6e-b1f5aa6bec45 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:08.509649 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b12c97-43de-4f41-bc6e-b1f5aa6bec45 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:04:08.509955 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b12c97-43de-4f41-bc6e-b1f5aa6bec45 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:08.511535 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b12c97-43de-4f41-bc6e-b1f5aa6bec45 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:08.524133 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1388/2767] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jun 15 15:04:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:04:08.529753 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0bb9f5-57ee-46b4-aac6-879b1289f026 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:08.540664 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0bb9f5-57ee-46b4-aac6-879b1289f026 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:08.546051 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6285ef7a-7fdd-48aa-9d1a-31551aee9406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:08.546716 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6285ef7a-7fdd-48aa-9d1a-31551aee9406 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:08.547002 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6285ef7a-7fdd-48aa-9d1a-31551aee9406 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:08.547284 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6285ef7a-7fdd-48aa-9d1a-31551aee9406 admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:08.548821 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6285ef7a-7fdd-48aa-9d1a-31551aee9406 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:08.550727 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6285ef7a-7fdd-48aa-9d1a-31551aee9406 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:08.652156 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1380/2768] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 15:04:08 2022] POST /identity/v3/projects => generated 381 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:04:08.656689 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b12c97-43de-4f41-bc6e-b1f5aa6bec45 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:08.662472 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:04:08.667812 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b12c97-43de-4f41-bc6e-b1f5aa6bec45 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:08.726614 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4ba21b-26ae-45a3-bea4-225ae2f1ea89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:08.727313 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4ba21b-26ae-45a3-bea4-225ae2f1ea89 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:08.727625 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4ba21b-26ae-45a3-bea4-225ae2f1ea89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:08.727904 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4ba21b-26ae-45a3-bea4-225ae2f1ea89 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:08.728678 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4ba21b-26ae-45a3-bea4-225ae2f1ea89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:08.730785 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4ba21b-26ae-45a3-bea4-225ae2f1ea89 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:08.851499 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1389/2769] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 15:04:08 2022] POST /identity/v3/users => generated 291 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:04:08.855616 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6285ef7a-7fdd-48aa-9d1a-31551aee9406 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:08.866567 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6285ef7a-7fdd-48aa-9d1a-31551aee9406 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:08.871455 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3e2046-b571-4430-943d-d5a390f7c5fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:08.872159 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3e2046-b571-4430-943d-d5a390f7c5fd admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:08.872448 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3e2046-b571-4430-943d-d5a390f7c5fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:08.872984 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3e2046-b571-4430-943d-d5a390f7c5fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:08.873713 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3e2046-b571-4430-943d-d5a390f7c5fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:08.875352 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3e2046-b571-4430-943d-d5a390f7c5fd admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:08.886454 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1381/2770] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 15:04:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:04:08.890258 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4ba21b-26ae-45a3-bea4-225ae2f1ea89 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:08.901381 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4ba21b-26ae-45a3-bea4-225ae2f1ea89 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:08.906449 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa8c706-5d27-440b-94ef-a22cc806f843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:08.907854 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8c706-5d27-440b-94ef-a22cc806f843 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:08.908396 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8c706-5d27-440b-94ef-a22cc806f843 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:08.908688 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8c706-5d27-440b-94ef-a22cc806f843 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:08.909422 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa8c706-5d27-440b-94ef-a22cc806f843 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:08.911048 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa8c706-5d27-440b-94ef-a22cc806f843 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:08.921691 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1390/2771] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 15:04:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:04:08.925832 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3e2046-b571-4430-943d-d5a390f7c5fd admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:08.936295 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3e2046-b571-4430-943d-d5a390f7c5fd admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:08.941400 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ddbbb9-026d-4add-a1c2-607696bc5cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:08.942284 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ddbbb9-026d-4add-a1c2-607696bc5cef admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:08.942563 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ddbbb9-026d-4add-a1c2-607696bc5cef admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:08.942838 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ddbbb9-026d-4add-a1c2-607696bc5cef admin admin] PATH_INFO: `/v3/projects/3efcbca3db4f49c6a11803516941ebe1/users/ae775f33122c48fabd917accd0e6cb16/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:08.958630 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ddbbb9-026d-4add-a1c2-607696bc5cef admin admin] RBAC: Authorizing `identity:create_grant(project_id=3efcbca3db4f49c6a11803516941ebe1, user_id=ae775f33122c48fabd917accd0e6cb16, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:08.962668 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ddbbb9-026d-4add-a1c2-607696bc5cef admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:09.061624 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1382/2772] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 15:04:08 2022] PUT /identity/v3/projects/3efcbca3db4f49c6a11803516941ebe1/users/ae775f33122c48fabd917accd0e6cb16/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:04:09.066816 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd1fd2f-9511-47aa-8ca8-6adb881b9f51 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:09.067103 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd1fd2f-9511-47aa-8ca8-6adb881b9f51 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:09.067392 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd1fd2f-9511-47aa-8ca8-6adb881b9f51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:09.086356 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-8bd1fd2f-9511-47aa-8ca8-6adb881b9f51 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:04:09.087607 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-8bd1fd2f-9511-47aa-8ca8-6adb881b9f51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:09.148291 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1391/2773] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:04:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:04:09.158519 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ddbbb9-026d-4add-a1c2-607696bc5cef admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:09.160261 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-38ddbbb9-026d-4add-a1c2-607696bc5cef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:09.182557 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:04:09.219032 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ddbbb9-026d-4add-a1c2-607696bc5cef admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:09.223603 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7690b14-c8a4-4446-931a-3961ba22c643 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:09.224298 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7690b14-c8a4-4446-931a-3961ba22c643 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:09.224613 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7690b14-c8a4-4446-931a-3961ba22c643 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:09.224897 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7690b14-c8a4-4446-931a-3961ba22c643 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:09.234763 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7690b14-c8a4-4446-931a-3961ba22c643 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:09.236227 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7690b14-c8a4-4446-931a-3961ba22c643 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:09.246574 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1383/2774] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:04:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 15:04:10.377315 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90b3cf9-a029-4e7c-a3d5-7c3110071120 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:10.378138 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90b3cf9-a029-4e7c-a3d5-7c3110071120 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:10.378484 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90b3cf9-a029-4e7c-a3d5-7c3110071120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:10.411688 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-c90b3cf9-a029-4e7c-a3d5-7c3110071120 None None] MFA Rules not processed for user `ae775f33122c48fabd917accd0e6cb16`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:04:10.412986 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c90b3cf9-a029-4e7c-a3d5-7c3110071120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:10.500057 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1392/2775] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:04:10 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:04:10.511933 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7690b14-c8a4-4446-931a-3961ba22c643 service neutron] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:10.513996 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-e7690b14-c8a4-4446-931a-3961ba22c643 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:10.572676 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7690b14-c8a4-4446-931a-3961ba22c643 service neutron] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:10.576758 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e083108-b0d7-4438-b770-7606431685cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:10.577411 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e083108-b0d7-4438-b770-7606431685cc service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:10.577696 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e083108-b0d7-4438-b770-7606431685cc service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:10.577978 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e083108-b0d7-4438-b770-7606431685cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:10.587608 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e083108-b0d7-4438-b770-7606431685cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:10.592652 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e083108-b0d7-4438-b770-7606431685cc service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:10.598896 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1384/2776] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:04:10 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 15:04:13.341633 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c63ccbb-6b9c-4642-848f-7b9a977c9989 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:13.342170 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c63ccbb-6b9c-4642-848f-7b9a977c9989 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:13.342170 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c63ccbb-6b9c-4642-848f-7b9a977c9989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:13.361409 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-6c63ccbb-6b9c-4642-848f-7b9a977c9989 None None] MFA Rules not processed for user `ae775f33122c48fabd917accd0e6cb16`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:04:13.362626 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-6c63ccbb-6b9c-4642-848f-7b9a977c9989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:13.369490 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1393/2777] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:04:13 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:04:13.609714 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e083108-b0d7-4438-b770-7606431685cc service cinder] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:13.612055 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-5e083108-b0d7-4438-b770-7606431685cc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:13.624868 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e083108-b0d7-4438-b770-7606431685cc service cinder] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:13.629147 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead61dd3-b454-4b4e-b7c4-95ef8eef7b33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:13.629779 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead61dd3-b454-4b4e-b7c4-95ef8eef7b33 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:13.630058 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead61dd3-b454-4b4e-b7c4-95ef8eef7b33 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:13.630333 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead61dd3-b454-4b4e-b7c4-95ef8eef7b33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:13.632075 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-ead61dd3-b454-4b4e-b7c4-95ef8eef7b33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:13.641957 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead61dd3-b454-4b4e-b7c4-95ef8eef7b33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:13.643401 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead61dd3-b454-4b4e-b7c4-95ef8eef7b33 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:13.651853 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1385/2778] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:04:13 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:04:23.354300 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=95,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:04:23.647986 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=49 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:21.275554 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c63ccbb-6b9c-4642-848f-7b9a977c9989 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:21.285865 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c63ccbb-6b9c-4642-848f-7b9a977c9989 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:21.289631 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff289ae-194a-41e3-b39a-bbe4b94b4197 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:21.290260 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff289ae-194a-41e3-b39a-bbe4b94b4197 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:21.290538 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff289ae-194a-41e3-b39a-bbe4b94b4197 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:21.290808 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff289ae-194a-41e3-b39a-bbe4b94b4197 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:21.292570 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-6ff289ae-194a-41e3-b39a-bbe4b94b4197 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:21.364536 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff289ae-194a-41e3-b39a-bbe4b94b4197 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:21.366005 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff289ae-194a-41e3-b39a-bbe4b94b4197 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:21.377576 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1394/2779] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:05:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:05:22.554077 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead61dd3-b454-4b4e-b7c4-95ef8eef7b33 service neutron] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:22.562952 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead61dd3-b454-4b4e-b7c4-95ef8eef7b33 service neutron] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:22.567272 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2153716b-a3ce-48d2-8cbb-0c050b8f9852 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:22.568111 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2153716b-a3ce-48d2-8cbb-0c050b8f9852 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:22.568404 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2153716b-a3ce-48d2-8cbb-0c050b8f9852 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:22.568685 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2153716b-a3ce-48d2-8cbb-0c050b8f9852 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:22.570469 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-2153716b-a3ce-48d2-8cbb-0c050b8f9852 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:22.585000 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2153716b-a3ce-48d2-8cbb-0c050b8f9852 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:22.586416 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2153716b-a3ce-48d2-8cbb-0c050b8f9852 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:22.598068 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1386/2780] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:05:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:05:26.102358 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba17206-38a8-4218-840e-38793f678e8e None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:26.102881 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba17206-38a8-4218-840e-38793f678e8e None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:26.103172 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba17206-38a8-4218-840e-38793f678e8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:26.126654 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-6ba17206-38a8-4218-840e-38793f678e8e None None] MFA Rules not processed for user `e209ae2f7fa7412b95c7c43c74c14426`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:05:26.126654 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-6ba17206-38a8-4218-840e-38793f678e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:26.181809 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1395/2781] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:05:26 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:05:32.592489 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=6 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:36.173565 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=59 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:44.730314 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdd14b6-9dcd-498a-9612-e71e13d8eb22 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:44.730886 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdd14b6-9dcd-498a-9612-e71e13d8eb22 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:44.730886 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdd14b6-9dcd-498a-9612-e71e13d8eb22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:44.748538 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-0fdd14b6-9dcd-498a-9612-e71e13d8eb22 None None] MFA Rules not processed for user `ae775f33122c48fabd917accd0e6cb16`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:05:44.749752 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-0fdd14b6-9dcd-498a-9612-e71e13d8eb22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:44.756790 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1387/2782] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:05:44 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:05:48.961376 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba17206-38a8-4218-840e-38793f678e8e None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:48.971369 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba17206-38a8-4218-840e-38793f678e8e None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:48.975423 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c0cd39-e4c5-4974-8e1f-b753df31dd5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:48.976361 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c0cd39-e4c5-4974-8e1f-b753df31dd5d service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:48.976771 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c0cd39-e4c5-4974-8e1f-b753df31dd5d service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:48.977056 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c0cd39-e4c5-4974-8e1f-b753df31dd5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:48.979266 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-86c0cd39-e4c5-4974-8e1f-b753df31dd5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:48.989930 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c0cd39-e4c5-4974-8e1f-b753df31dd5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:48.991450 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c0cd39-e4c5-4974-8e1f-b753df31dd5d service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:49.001252 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1396/2783] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:05:48 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:05:54.322199 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdd14b6-9dcd-498a-9612-e71e13d8eb22 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:54.323693 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-0fdd14b6-9dcd-498a-9612-e71e13d8eb22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:54.340505 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdd14b6-9dcd-498a-9612-e71e13d8eb22 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:54.346129 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f7a548-12ad-4e46-a127-b89b9bc5ff49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0927c05c340c48e48b18754888aa9c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:54.346818 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f7a548-12ad-4e46-a127-b89b9bc5ff49 service nova] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:54.347132 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f7a548-12ad-4e46-a127-b89b9bc5ff49 service nova] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:54.347416 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f7a548-12ad-4e46-a127-b89b9bc5ff49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:54.349271 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-30f7a548-12ad-4e46-a127-b89b9bc5ff49 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:54.362400 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f7a548-12ad-4e46-a127-b89b9bc5ff49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:54.363998 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f7a548-12ad-4e46-a127-b89b9bc5ff49 service nova] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:54.375607 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1388/2784] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:05:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:05:59.000843 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=6 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:59.102912 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ef1d1-44d7-42c9-877d-2f689ece379b None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:59.103200 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ef1d1-44d7-42c9-877d-2f689ece379b None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:59.103476 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ef1d1-44d7-42c9-877d-2f689ece379b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:59.123537 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-634ef1d1-44d7-42c9-877d-2f689ece379b None None] MFA Rules not processed for user `17f8c496bb0049719cd67c457ba40f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:05:59.124858 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-634ef1d1-44d7-42c9-877d-2f689ece379b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:59.182953 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1397/2785] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:05:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:06:04.370367 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=9 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:06:04.558247 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f7a548-12ad-4e46-a127-b89b9bc5ff49 service nova] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:06:04.559410 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-30f7a548-12ad-4e46-a127-b89b9bc5ff49 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:06:04.577805 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f7a548-12ad-4e46-a127-b89b9bc5ff49 service nova] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:06:04.586723 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ed0c8d-fbdd-4482-a88a-1031c49c408f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0927c05c340c48e48b18754888aa9c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:06:04.586723 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ed0c8d-fbdd-4482-a88a-1031c49c408f service nova] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:06:04.586723 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ed0c8d-fbdd-4482-a88a-1031c49c408f service nova] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:06:04.586723 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ed0c8d-fbdd-4482-a88a-1031c49c408f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:06:04.588504 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-60ed0c8d-fbdd-4482-a88a-1031c49c408f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:06:04.603029 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ed0c8d-fbdd-4482-a88a-1031c49c408f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:06:04.605326 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ed0c8d-fbdd-4482-a88a-1031c49c408f service nova] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:06:04.618762 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1389/2786] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:06:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:06:09.176020 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=25 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:06:14.613328 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=6 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:07:08.202398 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634ef1d1-44d7-42c9-877d-2f689ece379b None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:07:08.203845 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-634ef1d1-44d7-42c9-877d-2f689ece379b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:07:08.282261 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634ef1d1-44d7-42c9-877d-2f689ece379b None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:07:08.287523 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeed139c-3363-4c04-812f-34984397c876 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bb6671c19464424a6e9cd4519ac93e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:07:08.288356 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed139c-3363-4c04-812f-34984397c876 service placement] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:07:08.288643 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed139c-3363-4c04-812f-34984397c876 service placement] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:07:08.288921 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed139c-3363-4c04-812f-34984397c876 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:07:08.290787 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-eeed139c-3363-4c04-812f-34984397c876 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:07:08.303862 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeed139c-3363-4c04-812f-34984397c876 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:07:08.305716 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeed139c-3363-4c04-812f-34984397c876 service placement] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:07:08.316558 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1398/2787] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:07:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:07:08.335999 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ed0c8d-fbdd-4482-a88a-1031c49c408f service nova] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:07:08.345676 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ed0c8d-fbdd-4482-a88a-1031c49c408f service nova] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:07:08.369139 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0c5e7e-433a-4759-8f0a-78369e6b22f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:07:08.369139 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0c5e7e-433a-4759-8f0a-78369e6b22f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:07:08.369139 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0c5e7e-433a-4759-8f0a-78369e6b22f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:07:08.369139 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0c5e7e-433a-4759-8f0a-78369e6b22f3 admin admin] PATH_INFO: `/v3/users/ae775f33122c48fabd917accd0e6cb16` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:07:08.369542 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0c5e7e-433a-4759-8f0a-78369e6b22f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae775f33122c48fabd917accd0e6cb16)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:07:08.369542 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0c5e7e-433a-4759-8f0a-78369e6b22f3 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:07:08.551198 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-be0c5e7e-433a-4759-8f0a-78369e6b22f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae775f33122c48fabd917accd0e6cb16'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:07:08.551859 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-be0c5e7e-433a-4759-8f0a-78369e6b22f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae775f33122c48fabd917accd0e6cb16'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:07:08.559820 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-be0c5e7e-433a-4759-8f0a-78369e6b22f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae775f33122c48fabd917accd0e6cb16'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:07:08.560495 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-be0c5e7e-433a-4759-8f0a-78369e6b22f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae775f33122c48fabd917accd0e6cb16'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:07:08.602776 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-be0c5e7e-433a-4759-8f0a-78369e6b22f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae775f33122c48fabd917accd0e6cb16'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:07:08.621348 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1390/2788] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 15:07:08 2022] DELETE /identity/v3/users/ae775f33122c48fabd917accd0e6cb16 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:07:08.850624 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeed139c-3363-4c04-812f-34984397c876 service placement] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:07:08.865504 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeed139c-3363-4c04-812f-34984397c876 service placement] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:07:08.926540 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bbc4b4-2655-4e70-8f2e-df3007d2c081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:07:08.927446 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bbc4b4-2655-4e70-8f2e-df3007d2c081 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:07:08.927730 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bbc4b4-2655-4e70-8f2e-df3007d2c081 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:07:08.928005 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bbc4b4-2655-4e70-8f2e-df3007d2c081 admin admin] PATH_INFO: `/v3/projects/3efcbca3db4f49c6a11803516941ebe1` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:07:08.929887 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bbc4b4-2655-4e70-8f2e-df3007d2c081 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3efcbca3db4f49c6a11803516941ebe1)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:07:08.931773 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bbc4b4-2655-4e70-8f2e-df3007d2c081 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:07:09.048017 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-54bbc4b4-2655-4e70-8f2e-df3007d2c081 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3efcbca3db4f49c6a11803516941ebe1', 'request_id': 'req-54bbc4b4-2655-4e70-8f2e-df3007d2c081'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:07:09.054741 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-54bbc4b4-2655-4e70-8f2e-df3007d2c081 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3efcbca3db4f49c6a11803516941ebe1', 'request_id': 'req-54bbc4b4-2655-4e70-8f2e-df3007d2c081'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:07:09.151786 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-54bbc4b4-2655-4e70-8f2e-df3007d2c081 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3efcbca3db4f49c6a11803516941ebe1', 'request_id': 'req-54bbc4b4-2655-4e70-8f2e-df3007d2c081'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:07:09.163187 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-54bbc4b4-2655-4e70-8f2e-df3007d2c081 admin admin] The token cache is being invalidate because project 3efcbca3db4f49c6a11803516941ebe1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:07:09.163499 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-54bbc4b4-2655-4e70-8f2e-df3007d2c081 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:07:09.164418 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-54bbc4b4-2655-4e70-8f2e-df3007d2c081 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:07:09.166050 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1399/2789] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 15:07:08 2022] DELETE /identity/v3/projects/3efcbca3db4f49c6a11803516941ebe1 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:07:18.617201 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=29,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:07:19.159612 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=83,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:08:11.554045 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0c5e7e-433a-4759-8f0a-78369e6b22f3 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:08:11.555436 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-be0c5e7e-433a-4759-8f0a-78369e6b22f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:08:11.612663 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0c5e7e-433a-4759-8f0a-78369e6b22f3 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:08:11.660662 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30042f48-a424-43aa-bdc3-50f915beed4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:08:11.661338 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30042f48-a424-43aa-bdc3-50f915beed4d service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:08:11.661620 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30042f48-a424-43aa-bdc3-50f915beed4d service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:08:11.661896 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30042f48-a424-43aa-bdc3-50f915beed4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:08:11.663651 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-30042f48-a424-43aa-bdc3-50f915beed4d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:08:11.721155 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30042f48-a424-43aa-bdc3-50f915beed4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:08:11.722592 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30042f48-a424-43aa-bdc3-50f915beed4d service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:08:11.731656 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1391/2790] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:08:11 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:08:17.818999 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bbc4b4-2655-4e70-8f2e-df3007d2c081 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:08:17.820553 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-54bbc4b4-2655-4e70-8f2e-df3007d2c081 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:08:17.895955 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bbc4b4-2655-4e70-8f2e-df3007d2c081 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:08:17.901125 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3946887f-7ec4-48fb-84d3-cbb87142a4c0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bb6671c19464424a6e9cd4519ac93e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:08:17.901784 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3946887f-7ec4-48fb-84d3-cbb87142a4c0 service placement] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:08:17.902067 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3946887f-7ec4-48fb-84d3-cbb87142a4c0 service placement] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:08:17.902344 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3946887f-7ec4-48fb-84d3-cbb87142a4c0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:08:17.905923 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-3946887f-7ec4-48fb-84d3-cbb87142a4c0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:08:17.923610 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3946887f-7ec4-48fb-84d3-cbb87142a4c0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:08:17.925500 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3946887f-7ec4-48fb-84d3-cbb87142a4c0 service placement] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:08:17.938450 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1400/2791] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:08:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:08:21.727852 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=68 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:08:22.272933 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc69a4bd-3c02-4dc6-9aa9-1e8523d49699 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:08:22.273219 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc69a4bd-3c02-4dc6-9aa9-1e8523d49699 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:08:22.273504 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc69a4bd-3c02-4dc6-9aa9-1e8523d49699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:08:22.291330 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-fc69a4bd-3c02-4dc6-9aa9-1e8523d49699 None None] MFA Rules not processed for user `17f8c496bb0049719cd67c457ba40f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:08:22.292574 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-fc69a4bd-3c02-4dc6-9aa9-1e8523d49699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:08:22.349510 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1392/2792] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:08:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:08:24.074255 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3946887f-7ec4-48fb-84d3-cbb87142a4c0 service placement] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:08:24.085476 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3946887f-7ec4-48fb-84d3-cbb87142a4c0 service placement] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:08:24.090605 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19697ef-e608-4231-876a-fb2524de3403 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bb6671c19464424a6e9cd4519ac93e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:08:24.091240 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19697ef-e608-4231-876a-fb2524de3403 service placement] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:08:24.091527 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19697ef-e608-4231-876a-fb2524de3403 service placement] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:08:24.091808 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19697ef-e608-4231-876a-fb2524de3403 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:08:24.094186 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-f19697ef-e608-4231-876a-fb2524de3403 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:08:24.106114 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19697ef-e608-4231-876a-fb2524de3403 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:08:24.107686 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19697ef-e608-4231-876a-fb2524de3403 service placement] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:08:24.118094 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1401/2793] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:08:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:08:24.394559 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59437ed0-0e93-4860-b89a-66698ac1d766 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:08:24.395073 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59437ed0-0e93-4860-b89a-66698ac1d766 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:08:24.395206 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59437ed0-0e93-4860-b89a-66698ac1d766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:08:24.412445 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-59437ed0-0e93-4860-b89a-66698ac1d766 None None] MFA Rules not processed for user `e209ae2f7fa7412b95c7c43c74c14426`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:08:24.413628 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-59437ed0-0e93-4860-b89a-66698ac1d766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:08:24.420350 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1393/2794] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:08:24 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:08:26.291096 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19697ef-e608-4231-876a-fb2524de3403 service placement] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:08:26.292626 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-f19697ef-e608-4231-876a-fb2524de3403 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:08:26.369393 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19697ef-e608-4231-876a-fb2524de3403 service placement] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:08:26.375100 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df56b576-d396-4676-a0a1-2618fb2f24c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0927c05c340c48e48b18754888aa9c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:08:26.379082 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df56b576-d396-4676-a0a1-2618fb2f24c6 service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:08:26.379082 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df56b576-d396-4676-a0a1-2618fb2f24c6 service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:08:26.379082 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df56b576-d396-4676-a0a1-2618fb2f24c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:08:26.379082 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-df56b576-d396-4676-a0a1-2618fb2f24c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:08:26.388699 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df56b576-d396-4676-a0a1-2618fb2f24c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:08:26.391059 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df56b576-d396-4676-a0a1-2618fb2f24c6 service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:08:26.401173 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1402/2795] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:08:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:08:34.407103 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=28 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:08:36.396620 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=72 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:08:38.196504 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59437ed0-0e93-4860-b89a-66698ac1d766 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:08:38.197977 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-59437ed0-0e93-4860-b89a-66698ac1d766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:08:38.215110 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59437ed0-0e93-4860-b89a-66698ac1d766 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:08:38.221801 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce42682-81f8-4d43-9f87-f651e6b28c45 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0927c05c340c48e48b18754888aa9c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:08:38.222573 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce42682-81f8-4d43-9f87-f651e6b28c45 service nova] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:08:38.222853 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce42682-81f8-4d43-9f87-f651e6b28c45 service nova] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:08:38.223214 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce42682-81f8-4d43-9f87-f651e6b28c45 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:08:38.224907 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-dce42682-81f8-4d43-9f87-f651e6b28c45 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:08:38.239187 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce42682-81f8-4d43-9f87-f651e6b28c45 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:08:38.240905 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce42682-81f8-4d43-9f87-f651e6b28c45 service nova] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:08:38.252052 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1394/2796] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:08:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:08:48.247415 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=6 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:08:51.266764 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df56b576-d396-4676-a0a1-2618fb2f24c6 service nova] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:08:51.268245 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-df56b576-d396-4676-a0a1-2618fb2f24c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:08:51.328313 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df56b576-d396-4676-a0a1-2618fb2f24c6 service nova] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:08:51.331958 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7733b73-01ec-4ed3-9571-3b56ed992aee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:08:51.332781 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7733b73-01ec-4ed3-9571-3b56ed992aee service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:08:51.332940 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7733b73-01ec-4ed3-9571-3b56ed992aee service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:08:51.333204 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7733b73-01ec-4ed3-9571-3b56ed992aee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:08:51.335005 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a7733b73-01ec-4ed3-9571-3b56ed992aee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:08:51.398606 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7733b73-01ec-4ed3-9571-3b56ed992aee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:08:51.400385 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7733b73-01ec-4ed3-9571-3b56ed992aee service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:08:51.411164 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1403/2797] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:08:51 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:09:01.406435 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=50 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:10:22.432684 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce42682-81f8-4d43-9f87-f651e6b28c45 service nova] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:10:22.434201 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-dce42682-81f8-4d43-9f87-f651e6b28c45 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:10:22.446464 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce42682-81f8-4d43-9f87-f651e6b28c45 service nova] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:10:22.450044 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc245bc-f458-47d0-ba6f-5515c81ffd35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:10:22.451332 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc245bc-f458-47d0-ba6f-5515c81ffd35 service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:10:22.451634 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc245bc-f458-47d0-ba6f-5515c81ffd35 service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:10:22.451914 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc245bc-f458-47d0-ba6f-5515c81ffd35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:10:22.453697 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-7bc245bc-f458-47d0-ba6f-5515c81ffd35 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:10:22.466702 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc245bc-f458-47d0-ba6f-5515c81ffd35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:10:22.468132 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc245bc-f458-47d0-ba6f-5515c81ffd35 service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:10:22.479309 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1395/2798] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:10:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:10:22.643719 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7733b73-01ec-4ed3-9571-3b56ed992aee service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:10:22.654529 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7733b73-01ec-4ed3-9571-3b56ed992aee service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:10:22.658340 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce37274-5358-4727-8d01-98028f0f6090 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:10:22.658995 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce37274-5358-4727-8d01-98028f0f6090 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:10:22.659287 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce37274-5358-4727-8d01-98028f0f6090 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:10:22.659570 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce37274-5358-4727-8d01-98028f0f6090 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:10:22.661450 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-5ce37274-5358-4727-8d01-98028f0f6090 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:10:22.676796 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce37274-5358-4727-8d01-98028f0f6090 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:10:22.678234 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce37274-5358-4727-8d01-98028f0f6090 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:10:22.689666 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1404/2799] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:10:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:10:32.474416 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=6 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:10:32.684510 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=6 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:10:34.729939 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18304a1-f802-4abb-bb89-24e48ba01e2e None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:10:34.730475 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18304a1-f802-4abb-bb89-24e48ba01e2e None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:10:34.730583 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18304a1-f802-4abb-bb89-24e48ba01e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:10:34.748469 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-a18304a1-f802-4abb-bb89-24e48ba01e2e None None] MFA Rules not processed for user `17f8c496bb0049719cd67c457ba40f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:10:34.749589 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-a18304a1-f802-4abb-bb89-24e48ba01e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:10:34.756480 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1396/2800] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:10:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:10:35.057709 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce37274-5358-4727-8d01-98028f0f6090 service cinder] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:10:35.059160 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-5ce37274-5358-4727-8d01-98028f0f6090 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:10:35.070779 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce37274-5358-4727-8d01-98028f0f6090 service cinder] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:10:35.074302 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22842597-99b4-43e1-9415-6c93575591d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:10:35.074928 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22842597-99b4-43e1-9415-6c93575591d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:10:35.075211 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22842597-99b4-43e1-9415-6c93575591d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:10:35.075680 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22842597-99b4-43e1-9415-6c93575591d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:10:35.077419 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-22842597-99b4-43e1-9415-6c93575591d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:10:35.086944 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22842597-99b4-43e1-9415-6c93575591d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:10:35.088386 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22842597-99b4-43e1-9415-6c93575591d0 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:10:35.096963 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1405/2801] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:10:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:10:44.742779 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:10:45.093442 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=6 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:10:48.347898 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18304a1-f802-4abb-bb89-24e48ba01e2e None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:10:48.358928 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18304a1-f802-4abb-bb89-24e48ba01e2e None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:10:48.362600 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74931e9-ae37-4a3c-a99b-b4d49bcb1ccd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:10:48.363187 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74931e9-ae37-4a3c-a99b-b4d49bcb1ccd service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:10:48.363567 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74931e9-ae37-4a3c-a99b-b4d49bcb1ccd service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:10:48.363856 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74931e9-ae37-4a3c-a99b-b4d49bcb1ccd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:10:48.365695 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-d74931e9-ae37-4a3c-a99b-b4d49bcb1ccd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:10:48.378276 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74931e9-ae37-4a3c-a99b-b4d49bcb1ccd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:10:48.379770 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74931e9-ae37-4a3c-a99b-b4d49bcb1ccd service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:10:48.388782 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1397/2802] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:10:48 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:10:57.472655 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22842597-99b4-43e1-9415-6c93575591d0 service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:10:57.486641 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22842597-99b4-43e1-9415-6c93575591d0 service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:10:57.492328 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbca891-b072-4f41-892a-a34b693d7549 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0927c05c340c48e48b18754888aa9c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:10:57.492971 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbca891-b072-4f41-892a-a34b693d7549 service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:10:57.493260 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbca891-b072-4f41-892a-a34b693d7549 service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:10:57.493542 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbca891-b072-4f41-892a-a34b693d7549 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:10:57.495571 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-3dbca891-b072-4f41-892a-a34b693d7549 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:10:57.508793 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbca891-b072-4f41-892a-a34b693d7549 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:10:57.510487 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbca891-b072-4f41-892a-a34b693d7549 service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:10:57.522319 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1406/2803] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:10:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:10:58.384498 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=6 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:11:07.517173 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=6 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:11:09.259945 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74931e9-ae37-4a3c-a99b-b4d49bcb1ccd service neutron] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:11:09.268847 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74931e9-ae37-4a3c-a99b-b4d49bcb1ccd service neutron] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:11:09.272515 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bb9315-1558-4c5d-96c3-648644546f9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:11:09.273152 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb9315-1558-4c5d-96c3-648644546f9b service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:11:09.273439 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb9315-1558-4c5d-96c3-648644546f9b service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:11:09.273720 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb9315-1558-4c5d-96c3-648644546f9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:11:09.275476 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-d1bb9315-1558-4c5d-96c3-648644546f9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:11:09.340038 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bb9315-1558-4c5d-96c3-648644546f9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:11:09.341590 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bb9315-1558-4c5d-96c3-648644546f9b service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:11:09.351445 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1398/2804] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:11:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:11:12.725980 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbca891-b072-4f41-892a-a34b693d7549 service nova] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:11:12.727693 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-3dbca891-b072-4f41-892a-a34b693d7549 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:11:12.742677 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbca891-b072-4f41-892a-a34b693d7549 service nova] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:11:12.748826 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcc3a14-42ae-4a40-9959-b75686e7143b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bb6671c19464424a6e9cd4519ac93e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:11:12.749535 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcc3a14-42ae-4a40-9959-b75686e7143b service placement] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:11:12.749912 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcc3a14-42ae-4a40-9959-b75686e7143b service placement] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:11:12.750242 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcc3a14-42ae-4a40-9959-b75686e7143b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:11:12.752140 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-2bcc3a14-42ae-4a40-9959-b75686e7143b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:11:12.764871 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcc3a14-42ae-4a40-9959-b75686e7143b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:11:12.766407 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcc3a14-42ae-4a40-9959-b75686e7143b service placement] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:11:12.776895 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1407/2805] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:11:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 15:11:16.546844 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bb9315-1558-4c5d-96c3-648644546f9b service neutron] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:11:16.548419 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-d1bb9315-1558-4c5d-96c3-648644546f9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:11:16.562263 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bb9315-1558-4c5d-96c3-648644546f9b service neutron] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:11:16.569364 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb49c6b-dad5-4232-93fb-166eb074c303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:11:16.570199 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb49c6b-dad5-4232-93fb-166eb074c303 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:11:16.570481 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb49c6b-dad5-4232-93fb-166eb074c303 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:11:16.570755 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb49c6b-dad5-4232-93fb-166eb074c303 admin admin] PATH_INFO: `/v3/users/e209ae2f7fa7412b95c7c43c74c14426` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:11:16.585903 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb49c6b-dad5-4232-93fb-166eb074c303 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e209ae2f7fa7412b95c7c43c74c14426)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:11:16.589820 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb49c6b-dad5-4232-93fb-166eb074c303 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:11:16.767627 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-0cb49c6b-dad5-4232-93fb-166eb074c303 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e209ae2f7fa7412b95c7c43c74c14426'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:11:16.768355 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-0cb49c6b-dad5-4232-93fb-166eb074c303 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e209ae2f7fa7412b95c7c43c74c14426'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:11:16.775246 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-0cb49c6b-dad5-4232-93fb-166eb074c303 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e209ae2f7fa7412b95c7c43c74c14426'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:11:16.775861 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-0cb49c6b-dad5-4232-93fb-166eb074c303 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e209ae2f7fa7412b95c7c43c74c14426'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:11:16.862516 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-0cb49c6b-dad5-4232-93fb-166eb074c303 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e209ae2f7fa7412b95c7c43c74c14426'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:11:16.881455 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1399/2806] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 15:11:16 2022] DELETE /identity/v3/users/e209ae2f7fa7412b95c7c43c74c14426 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:11:17.180541 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcc3a14-42ae-4a40-9959-b75686e7143b service placement] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:11:17.193351 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcc3a14-42ae-4a40-9959-b75686e7143b service placement] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:11:17.253778 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d6fac4-eaca-4b4c-8f41-b65a627830ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:11:17.254673 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d6fac4-eaca-4b4c-8f41-b65a627830ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:11:17.254967 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d6fac4-eaca-4b4c-8f41-b65a627830ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:11:17.255269 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d6fac4-eaca-4b4c-8f41-b65a627830ee admin admin] PATH_INFO: `/v3/projects/79f361ea99aa45c185d50d83c84e5dc1` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:11:17.267497 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d6fac4-eaca-4b4c-8f41-b65a627830ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=79f361ea99aa45c185d50d83c84e5dc1)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:11:17.269711 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d6fac4-eaca-4b4c-8f41-b65a627830ee admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:11:17.407157 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-03d6fac4-eaca-4b4c-8f41-b65a627830ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79f361ea99aa45c185d50d83c84e5dc1', 'request_id': 'req-03d6fac4-eaca-4b4c-8f41-b65a627830ee'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:11:17.413906 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-03d6fac4-eaca-4b4c-8f41-b65a627830ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79f361ea99aa45c185d50d83c84e5dc1', 'request_id': 'req-03d6fac4-eaca-4b4c-8f41-b65a627830ee'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:11:17.476335 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-03d6fac4-eaca-4b4c-8f41-b65a627830ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79f361ea99aa45c185d50d83c84e5dc1', 'request_id': 'req-03d6fac4-eaca-4b4c-8f41-b65a627830ee'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:11:17.487664 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-03d6fac4-eaca-4b4c-8f41-b65a627830ee admin admin] The token cache is being invalidate because project 79f361ea99aa45c185d50d83c84e5dc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:11:17.487951 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-03d6fac4-eaca-4b4c-8f41-b65a627830ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:11:17.488619 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-03d6fac4-eaca-4b4c-8f41-b65a627830ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:11:17.490233 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1408/2807] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 15:11:17 2022] DELETE /identity/v3/projects/79f361ea99aa45c185d50d83c84e5dc1 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:11:17.494644 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb49c6b-dad5-4232-93fb-166eb074c303 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:11:17.496045 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-0cb49c6b-dad5-4232-93fb-166eb074c303 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:11:17.509754 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb49c6b-dad5-4232-93fb-166eb074c303 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:11:17.563312 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:11:17.570418 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12216a48-6a87-432c-93ad-6cfc1f648112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:11:17.571293 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12216a48-6a87-432c-93ad-6cfc1f648112 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:11:17.571576 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12216a48-6a87-432c-93ad-6cfc1f648112 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:11:17.571854 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12216a48-6a87-432c-93ad-6cfc1f648112 admin admin] PATH_INFO: `/v3/users/6bc416dafea64c64bf5d472e908daacb` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:11:17.573053 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12216a48-6a87-432c-93ad-6cfc1f648112 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bc416dafea64c64bf5d472e908daacb)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:11:17.575503 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12216a48-6a87-432c-93ad-6cfc1f648112 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:11:17.775439 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-12216a48-6a87-432c-93ad-6cfc1f648112 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bc416dafea64c64bf5d472e908daacb'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:11:17.776139 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-12216a48-6a87-432c-93ad-6cfc1f648112 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bc416dafea64c64bf5d472e908daacb'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:11:17.783578 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-12216a48-6a87-432c-93ad-6cfc1f648112 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bc416dafea64c64bf5d472e908daacb'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:11:17.784269 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-12216a48-6a87-432c-93ad-6cfc1f648112 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bc416dafea64c64bf5d472e908daacb'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:11:17.853457 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-12216a48-6a87-432c-93ad-6cfc1f648112 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bc416dafea64c64bf5d472e908daacb'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:11:17.873682 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1400/2808] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 15:11:17 2022] DELETE /identity/v3/users/6bc416dafea64c64bf5d472e908daacb => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:11:18.132631 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d6fac4-eaca-4b4c-8f41-b65a627830ee admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:11:18.134133 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-03d6fac4-eaca-4b4c-8f41-b65a627830ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:11:18.156983 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d6fac4-eaca-4b4c-8f41-b65a627830ee admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:11:18.217092 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a660199-4abd-4581-aca7-b8af1fc5a281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:11:18.217964 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a660199-4abd-4581-aca7-b8af1fc5a281 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:11:18.218245 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a660199-4abd-4581-aca7-b8af1fc5a281 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:11:18.218523 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a660199-4abd-4581-aca7-b8af1fc5a281 admin admin] PATH_INFO: `/v3/projects/0e3cf089909f446981d63c658e07666b` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:11:18.219735 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a660199-4abd-4581-aca7-b8af1fc5a281 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e3cf089909f446981d63c658e07666b)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:11:18.221666 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a660199-4abd-4581-aca7-b8af1fc5a281 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:11:18.232737 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:11:18.372395 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-3a660199-4abd-4581-aca7-b8af1fc5a281 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e3cf089909f446981d63c658e07666b', 'request_id': 'req-3a660199-4abd-4581-aca7-b8af1fc5a281'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:11:18.378904 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-3a660199-4abd-4581-aca7-b8af1fc5a281 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e3cf089909f446981d63c658e07666b', 'request_id': 'req-3a660199-4abd-4581-aca7-b8af1fc5a281'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:11:18.473638 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-3a660199-4abd-4581-aca7-b8af1fc5a281 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e3cf089909f446981d63c658e07666b', 'request_id': 'req-3a660199-4abd-4581-aca7-b8af1fc5a281'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:11:18.485858 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-3a660199-4abd-4581-aca7-b8af1fc5a281 admin admin] The token cache is being invalidate because project 0e3cf089909f446981d63c658e07666b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:11:18.486150 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-3a660199-4abd-4581-aca7-b8af1fc5a281 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:11:18.486787 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-3a660199-4abd-4581-aca7-b8af1fc5a281 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:11:18.488423 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1409/2809] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 15:11:18 2022] DELETE /identity/v3/projects/0e3cf089909f446981d63c658e07666b => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:11:27.868843 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=28,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:11:28.673764 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=20,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:12:27.041650 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12216a48-6a87-432c-93ad-6cfc1f648112 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:27.043072 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-12216a48-6a87-432c-93ad-6cfc1f648112 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:12:27.153023 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12216a48-6a87-432c-93ad-6cfc1f648112 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:27.202102 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cd7563-524c-4d75-81e0-2908828fcde7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:27.202743 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd7563-524c-4d75-81e0-2908828fcde7 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:27.203021 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd7563-524c-4d75-81e0-2908828fcde7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:27.203293 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd7563-524c-4d75-81e0-2908828fcde7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:27.205049 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-18cd7563-524c-4d75-81e0-2908828fcde7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:12:27.282987 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:12:27.307237 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cd7563-524c-4d75-81e0-2908828fcde7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:27.308690 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cd7563-524c-4d75-81e0-2908828fcde7 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:27.328130 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1401/2810] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:12:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:12:30.943449 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a660199-4abd-4581-aca7-b8af1fc5a281 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:30.945020 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-3a660199-4abd-4581-aca7-b8af1fc5a281 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:12:31.015577 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a660199-4abd-4581-aca7-b8af1fc5a281 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:31.020544 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09271db-8fa2-4404-810e-908c62a18bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:31.021549 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09271db-8fa2-4404-810e-908c62a18bc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:31.021855 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09271db-8fa2-4404-810e-908c62a18bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:31.022133 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09271db-8fa2-4404-810e-908c62a18bc0 admin admin] PATH_INFO: `/v3/users/17f8c496bb0049719cd67c457ba40f35` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:31.023380 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09271db-8fa2-4404-810e-908c62a18bc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17f8c496bb0049719cd67c457ba40f35)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:31.025343 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09271db-8fa2-4404-810e-908c62a18bc0 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:31.258311 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-e09271db-8fa2-4404-810e-908c62a18bc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17f8c496bb0049719cd67c457ba40f35'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:31.258962 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-e09271db-8fa2-4404-810e-908c62a18bc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17f8c496bb0049719cd67c457ba40f35'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:31.259563 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-e09271db-8fa2-4404-810e-908c62a18bc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17f8c496bb0049719cd67c457ba40f35'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:31.333547 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-e09271db-8fa2-4404-810e-908c62a18bc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17f8c496bb0049719cd67c457ba40f35'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:31.346090 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-e09271db-8fa2-4404-810e-908c62a18bc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17f8c496bb0049719cd67c457ba40f35'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:31.358058 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1410/2811] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 15:12:30 2022] DELETE /identity/v3/users/17f8c496bb0049719cd67c457ba40f35 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:12:31.601131 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cd7563-524c-4d75-81e0-2908828fcde7 service neutron] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:31.613663 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cd7563-524c-4d75-81e0-2908828fcde7 service neutron] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:31.672444 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a889b72-063a-4594-af57-198535f0546a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:31.673296 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a889b72-063a-4594-af57-198535f0546a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:31.673573 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a889b72-063a-4594-af57-198535f0546a admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:31.673844 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a889b72-063a-4594-af57-198535f0546a admin admin] PATH_INFO: `/v3/projects/e6829672292b4494931500de38f4d924` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:31.675023 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a889b72-063a-4594-af57-198535f0546a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6829672292b4494931500de38f4d924)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:31.677783 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a889b72-063a-4594-af57-198535f0546a admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:31.779547 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-8a889b72-063a-4594-af57-198535f0546a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6829672292b4494931500de38f4d924', 'request_id': 'req-8a889b72-063a-4594-af57-198535f0546a'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:31.853260 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-8a889b72-063a-4594-af57-198535f0546a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6829672292b4494931500de38f4d924', 'request_id': 'req-8a889b72-063a-4594-af57-198535f0546a'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:31.859072 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-8a889b72-063a-4594-af57-198535f0546a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6829672292b4494931500de38f4d924', 'request_id': 'req-8a889b72-063a-4594-af57-198535f0546a'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:31.871205 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-8a889b72-063a-4594-af57-198535f0546a admin admin] The token cache is being invalidate because project e6829672292b4494931500de38f4d924 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:12:31.871509 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-8a889b72-063a-4594-af57-198535f0546a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:31.872161 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-8a889b72-063a-4594-af57-198535f0546a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:31.873721 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1402/2812] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 15:12:31 2022] DELETE /identity/v3/projects/e6829672292b4494931500de38f4d924 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:12:31.878708 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01520d73-a723-40d5-b210-3faf1d3ee073 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:31.878998 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01520d73-a723-40d5-b210-3faf1d3ee073 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:31.880156 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01520d73-a723-40d5-b210-3faf1d3ee073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:31.938807 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-01520d73-a723-40d5-b210-3faf1d3ee073 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:12:31.940032 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-01520d73-a723-40d5-b210-3faf1d3ee073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:12:31.984835 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:12:32.000883 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1411/2813] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:12:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:12:32.009387 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2430c26-39ec-4b9a-a1ae-6551db1dfdcb None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:32.009675 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2430c26-39ec-4b9a-a1ae-6551db1dfdcb None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:32.009958 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2430c26-39ec-4b9a-a1ae-6551db1dfdcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:32.029383 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-e2430c26-39ec-4b9a-a1ae-6551db1dfdcb None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:12:32.030653 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-e2430c26-39ec-4b9a-a1ae-6551db1dfdcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:12:32.039154 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1403/2814] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:12:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:12:32.043388 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01520d73-a723-40d5-b210-3faf1d3ee073 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:32.054164 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01520d73-a723-40d5-b210-3faf1d3ee073 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:32.058871 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b10d3c-2368-44f4-a42c-f67f1a048259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:32.059533 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b10d3c-2368-44f4-a42c-f67f1a048259 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:32.059820 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b10d3c-2368-44f4-a42c-f67f1a048259 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:32.060125 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b10d3c-2368-44f4-a42c-f67f1a048259 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:32.060857 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b10d3c-2368-44f4-a42c-f67f1a048259 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:12:32.061177 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b10d3c-2368-44f4-a42c-f67f1a048259 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:32.062767 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b10d3c-2368-44f4-a42c-f67f1a048259 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:32.074607 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1412/2815] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jun 15 15:12:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:12:32.078866 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2430c26-39ec-4b9a-a1ae-6551db1dfdcb None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:32.088956 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2430c26-39ec-4b9a-a1ae-6551db1dfdcb None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:32.093900 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30142532-c1da-4a9f-87e7-97fca46381d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:32.095052 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30142532-c1da-4a9f-87e7-97fca46381d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:32.095338 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30142532-c1da-4a9f-87e7-97fca46381d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:32.095614 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30142532-c1da-4a9f-87e7-97fca46381d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:32.096340 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30142532-c1da-4a9f-87e7-97fca46381d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:32.098867 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30142532-c1da-4a9f-87e7-97fca46381d1 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:32.191865 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1404/2816] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 15:12:32 2022] POST /identity/v3/projects => generated 405 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:12:32.195983 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b10d3c-2368-44f4-a42c-f67f1a048259 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:32.206017 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b10d3c-2368-44f4-a42c-f67f1a048259 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:32.262281 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bcfcab-c2bb-4952-b343-e69f5ceae728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:32.262949 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bcfcab-c2bb-4952-b343-e69f5ceae728 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:32.263235 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bcfcab-c2bb-4952-b343-e69f5ceae728 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:32.263512 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bcfcab-c2bb-4952-b343-e69f5ceae728 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:32.264273 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bcfcab-c2bb-4952-b343-e69f5ceae728 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:32.266175 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bcfcab-c2bb-4952-b343-e69f5ceae728 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:32.336826 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1413/2817] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 15:12:32 2022] POST /identity/v3/users => generated 303 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:12:32.340938 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30142532-c1da-4a9f-87e7-97fca46381d1 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:32.352157 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30142532-c1da-4a9f-87e7-97fca46381d1 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:32.357068 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22db6eb-9d47-47cc-b95e-9533afb61e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:32.357729 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22db6eb-9d47-47cc-b95e-9533afb61e31 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:32.358011 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22db6eb-9d47-47cc-b95e-9533afb61e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:32.358287 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22db6eb-9d47-47cc-b95e-9533afb61e31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:32.359004 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22db6eb-9d47-47cc-b95e-9533afb61e31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:32.360615 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22db6eb-9d47-47cc-b95e-9533afb61e31 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:32.377731 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1405/2818] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 15:12:32 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:12:32.378959 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bcfcab-c2bb-4952-b343-e69f5ceae728 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:32.389984 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bcfcab-c2bb-4952-b343-e69f5ceae728 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:32.396607 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc66719-dd86-41ea-bfcb-aac480287280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:32.397396 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc66719-dd86-41ea-bfcb-aac480287280 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:32.397545 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc66719-dd86-41ea-bfcb-aac480287280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:32.397822 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc66719-dd86-41ea-bfcb-aac480287280 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:32.398551 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc66719-dd86-41ea-bfcb-aac480287280 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:32.400274 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc66719-dd86-41ea-bfcb-aac480287280 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:32.411347 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1414/2819] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 15:12:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:12:32.415709 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22db6eb-9d47-47cc-b95e-9533afb61e31 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:32.425631 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22db6eb-9d47-47cc-b95e-9533afb61e31 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:32.431262 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a4d841-bf86-4bd6-862a-62b303066f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:32.432168 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a4d841-bf86-4bd6-862a-62b303066f97 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:32.432455 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a4d841-bf86-4bd6-862a-62b303066f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:32.432735 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a4d841-bf86-4bd6-862a-62b303066f97 admin admin] PATH_INFO: `/v3/projects/a89e90c09c9f422eb3a43b8d0660ce94/users/b2f1737a33a547119ae3d431abb602d3/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:32.441234 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:12:32.448465 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a4d841-bf86-4bd6-862a-62b303066f97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a89e90c09c9f422eb3a43b8d0660ce94, user_id=b2f1737a33a547119ae3d431abb602d3, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:32.451027 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a4d841-bf86-4bd6-862a-62b303066f97 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:32.525007 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1406/2820] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 15:12:32 2022] PUT /identity/v3/projects/a89e90c09c9f422eb3a43b8d0660ce94/users/b2f1737a33a547119ae3d431abb602d3/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:12:32.536279 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c03546-c822-45b0-ace1-910c447e2136 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:32.536678 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c03546-c822-45b0-ace1-910c447e2136 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:32.536950 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c03546-c822-45b0-ace1-910c447e2136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:32.565649 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-72c03546-c822-45b0-ace1-910c447e2136 None None] MFA Rules not processed for user `b2f1737a33a547119ae3d431abb602d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:12:32.567671 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-72c03546-c822-45b0-ace1-910c447e2136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:12:32.637366 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1415/2821] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:12:32 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:12:32.641813 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a4d841-bf86-4bd6-862a-62b303066f97 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:32.704069 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a4d841-bf86-4bd6-862a-62b303066f97 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:32.709690 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8438e9-8742-43b8-94bd-1271a1fc12ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:32.710345 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8438e9-8742-43b8-94bd-1271a1fc12ce admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:32.710654 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8438e9-8742-43b8-94bd-1271a1fc12ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:32.710928 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8438e9-8742-43b8-94bd-1271a1fc12ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:32.711605 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8438e9-8742-43b8-94bd-1271a1fc12ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:32.713520 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8438e9-8742-43b8-94bd-1271a1fc12ce admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:32.799740 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1407/2822] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 15:12:32 2022] POST /identity/v3/projects => generated 405 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:12:32.804433 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c03546-c822-45b0-ace1-910c447e2136 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:32.815447 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c03546-c822-45b0-ace1-910c447e2136 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:32.856913 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:12:32.872423 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a75f94-b23d-4a24-bda7-ab3835850357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:32.873089 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a75f94-b23d-4a24-bda7-ab3835850357 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:32.873510 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a75f94-b23d-4a24-bda7-ab3835850357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:32.873646 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a75f94-b23d-4a24-bda7-ab3835850357 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:32.874335 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a75f94-b23d-4a24-bda7-ab3835850357 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:32.876272 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a75f94-b23d-4a24-bda7-ab3835850357 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:32.934050 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1416/2823] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 15:12:32 2022] POST /identity/v3/users => generated 309 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:12:32.938086 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8438e9-8742-43b8-94bd-1271a1fc12ce admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:32.948651 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8438e9-8742-43b8-94bd-1271a1fc12ce admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:32.953171 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db4ef8c-22d5-41ef-aa00-e02d370e6274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:32.953848 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db4ef8c-22d5-41ef-aa00-e02d370e6274 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:32.954186 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db4ef8c-22d5-41ef-aa00-e02d370e6274 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:32.954414 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db4ef8c-22d5-41ef-aa00-e02d370e6274 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:32.955135 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db4ef8c-22d5-41ef-aa00-e02d370e6274 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:32.956749 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db4ef8c-22d5-41ef-aa00-e02d370e6274 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:32.967738 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1408/2824] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 15:12:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:12:32.971913 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a75f94-b23d-4a24-bda7-ab3835850357 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:32.982216 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a75f94-b23d-4a24-bda7-ab3835850357 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:32.986780 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042523cf-c1a7-4ce7-bd8e-5e69bd1e6614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:32.987640 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042523cf-c1a7-4ce7-bd8e-5e69bd1e6614 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:32.987921 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042523cf-c1a7-4ce7-bd8e-5e69bd1e6614 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:32.988307 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042523cf-c1a7-4ce7-bd8e-5e69bd1e6614 admin admin] PATH_INFO: `/v3/projects/29fe25acd5a841f7b9d864cea1c59d66/users/2e71147a32a24d6b8018dc469778e288/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:33.003958 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042523cf-c1a7-4ce7-bd8e-5e69bd1e6614 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29fe25acd5a841f7b9d864cea1c59d66, user_id=2e71147a32a24d6b8018dc469778e288, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:33.006686 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042523cf-c1a7-4ce7-bd8e-5e69bd1e6614 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:33.110009 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1417/2825] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 15:12:32 2022] PUT /identity/v3/projects/29fe25acd5a841f7b9d864cea1c59d66/users/2e71147a32a24d6b8018dc469778e288/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:12:33.114912 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db4ef8c-22d5-41ef-aa00-e02d370e6274 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:33.125127 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db4ef8c-22d5-41ef-aa00-e02d370e6274 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:33.181271 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e6246e-b3f5-45d9-b2be-b5169b46929b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:33.181924 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e6246e-b3f5-45d9-b2be-b5169b46929b admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:33.182204 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e6246e-b3f5-45d9-b2be-b5169b46929b admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:33.182480 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e6246e-b3f5-45d9-b2be-b5169b46929b admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:33.183188 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e6246e-b3f5-45d9-b2be-b5169b46929b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:33.184788 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e6246e-b3f5-45d9-b2be-b5169b46929b admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:33.194512 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1409/2826] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 15:12:33 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:12:33.198673 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042523cf-c1a7-4ce7-bd8e-5e69bd1e6614 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:33.210106 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042523cf-c1a7-4ce7-bd8e-5e69bd1e6614 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:33.215199 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e70db3-fcdd-4e76-9ccd-3560fa4b75c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:33.216064 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e70db3-fcdd-4e76-9ccd-3560fa4b75c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:33.216380 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e70db3-fcdd-4e76-9ccd-3560fa4b75c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:33.216657 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e70db3-fcdd-4e76-9ccd-3560fa4b75c5 admin admin] PATH_INFO: `/v3/projects/29fe25acd5a841f7b9d864cea1c59d66/users/2e71147a32a24d6b8018dc469778e288/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:33.218832 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e70db3-fcdd-4e76-9ccd-3560fa4b75c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29fe25acd5a841f7b9d864cea1c59d66, user_id=2e71147a32a24d6b8018dc469778e288, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:33.221505 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e70db3-fcdd-4e76-9ccd-3560fa4b75c5 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:33.232282 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1418/2827] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 15:12:33 2022] PUT /identity/v3/projects/29fe25acd5a841f7b9d864cea1c59d66/users/2e71147a32a24d6b8018dc469778e288/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:12:33.249933 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4603f962-8757-4254-ae43-cc001f736b3e None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:33.250250 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4603f962-8757-4254-ae43-cc001f736b3e None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:33.250528 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4603f962-8757-4254-ae43-cc001f736b3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:33.276644 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-4603f962-8757-4254-ae43-cc001f736b3e None None] MFA Rules not processed for user `2e71147a32a24d6b8018dc469778e288`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:12:33.277832 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-4603f962-8757-4254-ae43-cc001f736b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:12:33.322187 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:12:33.345875 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1410/2828] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:12:33 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:12:33.357175 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e70db3-fcdd-4e76-9ccd-3560fa4b75c5 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:33.359038 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-10e70db3-fcdd-4e76-9ccd-3560fa4b75c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:12:33.417039 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e70db3-fcdd-4e76-9ccd-3560fa4b75c5 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:33.420799 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1604e25e-0563-48e4-a6ec-2b116c8ae06a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:33.421442 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1604e25e-0563-48e4-a6ec-2b116c8ae06a service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:33.421722 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1604e25e-0563-48e4-a6ec-2b116c8ae06a service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:33.422001 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1604e25e-0563-48e4-a6ec-2b116c8ae06a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:33.431463 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1604e25e-0563-48e4-a6ec-2b116c8ae06a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:33.432927 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1604e25e-0563-48e4-a6ec-2b116c8ae06a service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:33.442548 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1419/2829] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:12:33 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:12:33.594387 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4603f962-8757-4254-ae43-cc001f736b3e None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:33.659344 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4603f962-8757-4254-ae43-cc001f736b3e None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:33.664116 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed37ba1-402a-4bd5-8db4-e966516c8f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:33.665006 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed37ba1-402a-4bd5-8db4-e966516c8f5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:33.665291 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed37ba1-402a-4bd5-8db4-e966516c8f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:33.665568 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed37ba1-402a-4bd5-8db4-e966516c8f5b admin admin] PATH_INFO: `/v3/users/b2f1737a33a547119ae3d431abb602d3` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:33.666779 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed37ba1-402a-4bd5-8db4-e966516c8f5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2f1737a33a547119ae3d431abb602d3)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:33.668730 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed37ba1-402a-4bd5-8db4-e966516c8f5b admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:33.835308 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-3ed37ba1-402a-4bd5-8db4-e966516c8f5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2f1737a33a547119ae3d431abb602d3'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:33.835957 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-3ed37ba1-402a-4bd5-8db4-e966516c8f5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2f1737a33a547119ae3d431abb602d3'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:33.842800 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-3ed37ba1-402a-4bd5-8db4-e966516c8f5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2f1737a33a547119ae3d431abb602d3'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:33.845277 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-3ed37ba1-402a-4bd5-8db4-e966516c8f5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2f1737a33a547119ae3d431abb602d3'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:33.921966 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-3ed37ba1-402a-4bd5-8db4-e966516c8f5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2f1737a33a547119ae3d431abb602d3'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:33.941085 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1411/2830] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 15:12:33 2022] DELETE /identity/v3/users/b2f1737a33a547119ae3d431abb602d3 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:12:33.946494 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd3472c-cfd4-4416-a200-5929cc3fea7f None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:33.946894 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd3472c-cfd4-4416-a200-5929cc3fea7f None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:33.947014 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd3472c-cfd4-4416-a200-5929cc3fea7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:33.964428 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-7bd3472c-cfd4-4416-a200-5929cc3fea7f None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:12:33.965628 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-7bd3472c-cfd4-4416-a200-5929cc3fea7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:12:34.024004 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1420/2831] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:12:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:12:34.033676 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed37ba1-402a-4bd5-8db4-e966516c8f5b admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:34.035084 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-3ed37ba1-402a-4bd5-8db4-e966516c8f5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:12:34.094314 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed37ba1-402a-4bd5-8db4-e966516c8f5b admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:34.097868 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43fd473-be7c-491f-b041-89c4a80ab731 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:34.098508 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43fd473-be7c-491f-b041-89c4a80ab731 service neutron] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:34.098789 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43fd473-be7c-491f-b041-89c4a80ab731 service neutron] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:34.099064 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43fd473-be7c-491f-b041-89c4a80ab731 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:34.108753 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43fd473-be7c-491f-b041-89c4a80ab731 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:34.110190 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43fd473-be7c-491f-b041-89c4a80ab731 service neutron] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:34.119626 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1412/2832] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:12:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:12:34.548940 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd3472c-cfd4-4416-a200-5929cc3fea7f None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:34.550383 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-7bd3472c-cfd4-4416-a200-5929cc3fea7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:12:34.563836 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd3472c-cfd4-4416-a200-5929cc3fea7f None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:34.568020 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06888f2-33a6-4981-bf8c-154f0071f359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:34.569124 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06888f2-33a6-4981-bf8c-154f0071f359 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:34.569976 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06888f2-33a6-4981-bf8c-154f0071f359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:34.570369 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06888f2-33a6-4981-bf8c-154f0071f359 admin admin] PATH_INFO: `/v3/projects/a89e90c09c9f422eb3a43b8d0660ce94` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:34.571912 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06888f2-33a6-4981-bf8c-154f0071f359 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a89e90c09c9f422eb3a43b8d0660ce94)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:34.573853 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06888f2-33a6-4981-bf8c-154f0071f359 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:34.667496 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:12:34.676807 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-e06888f2-33a6-4981-bf8c-154f0071f359 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a89e90c09c9f422eb3a43b8d0660ce94', 'request_id': 'req-e06888f2-33a6-4981-bf8c-154f0071f359'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:34.684110 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-e06888f2-33a6-4981-bf8c-154f0071f359 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a89e90c09c9f422eb3a43b8d0660ce94', 'request_id': 'req-e06888f2-33a6-4981-bf8c-154f0071f359'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:34.767534 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-e06888f2-33a6-4981-bf8c-154f0071f359 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a89e90c09c9f422eb3a43b8d0660ce94', 'request_id': 'req-e06888f2-33a6-4981-bf8c-154f0071f359'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:34.778179 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-e06888f2-33a6-4981-bf8c-154f0071f359 admin admin] The token cache is being invalidate because project a89e90c09c9f422eb3a43b8d0660ce94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:12:34.778465 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-e06888f2-33a6-4981-bf8c-154f0071f359 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:34.779122 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-e06888f2-33a6-4981-bf8c-154f0071f359 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:34.780711 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1421/2833] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 15:12:34 2022] DELETE /identity/v3/projects/a89e90c09c9f422eb3a43b8d0660ce94 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:12:34.784777 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43fd473-be7c-491f-b041-89c4a80ab731 service neutron] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:34.790776 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:12:34.794570 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43fd473-be7c-491f-b041-89c4a80ab731 service neutron] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:34.851189 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ff35e8-86e1-45e9-8454-927649daad75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:34.852054 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff35e8-86e1-45e9-8454-927649daad75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:34.852369 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff35e8-86e1-45e9-8454-927649daad75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:34.857489 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff35e8-86e1-45e9-8454-927649daad75 admin admin] PATH_INFO: `/v3/users/2e71147a32a24d6b8018dc469778e288` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:34.857489 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ff35e8-86e1-45e9-8454-927649daad75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e71147a32a24d6b8018dc469778e288)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:34.857489 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ff35e8-86e1-45e9-8454-927649daad75 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:35.001575 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-f5ff35e8-86e1-45e9-8454-927649daad75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e71147a32a24d6b8018dc469778e288'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:35.002195 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-f5ff35e8-86e1-45e9-8454-927649daad75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e71147a32a24d6b8018dc469778e288'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:35.009339 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-f5ff35e8-86e1-45e9-8454-927649daad75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e71147a32a24d6b8018dc469778e288'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:35.009988 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-f5ff35e8-86e1-45e9-8454-927649daad75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e71147a32a24d6b8018dc469778e288'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:35.070374 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-f5ff35e8-86e1-45e9-8454-927649daad75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e71147a32a24d6b8018dc469778e288'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:35.087754 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1413/2834] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 15:12:34 2022] DELETE /identity/v3/users/2e71147a32a24d6b8018dc469778e288 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:12:35.480553 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06888f2-33a6-4981-bf8c-154f0071f359 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:35.482122 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e06888f2-33a6-4981-bf8c-154f0071f359 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:12:35.496327 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06888f2-33a6-4981-bf8c-154f0071f359 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:35.555028 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b048436f-0398-44aa-b037-59411df27e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:35.555932 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b048436f-0398-44aa-b037-59411df27e65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:35.556261 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b048436f-0398-44aa-b037-59411df27e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:35.556546 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b048436f-0398-44aa-b037-59411df27e65 admin admin] PATH_INFO: `/v3/projects/29fe25acd5a841f7b9d864cea1c59d66` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:35.557737 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b048436f-0398-44aa-b037-59411df27e65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29fe25acd5a841f7b9d864cea1c59d66)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:35.559663 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b048436f-0398-44aa-b037-59411df27e65 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:35.682770 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-b048436f-0398-44aa-b037-59411df27e65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29fe25acd5a841f7b9d864cea1c59d66', 'request_id': 'req-b048436f-0398-44aa-b037-59411df27e65'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:35.688985 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-b048436f-0398-44aa-b037-59411df27e65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29fe25acd5a841f7b9d864cea1c59d66', 'request_id': 'req-b048436f-0398-44aa-b037-59411df27e65'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:35.761813 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-b048436f-0398-44aa-b037-59411df27e65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29fe25acd5a841f7b9d864cea1c59d66', 'request_id': 'req-b048436f-0398-44aa-b037-59411df27e65'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:35.774020 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-b048436f-0398-44aa-b037-59411df27e65 admin admin] The token cache is being invalidate because project 29fe25acd5a841f7b9d864cea1c59d66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:12:35.774020 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-b048436f-0398-44aa-b037-59411df27e65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:35.774691 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-b048436f-0398-44aa-b037-59411df27e65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:35.776280 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1422/2835] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 15:12:35 2022] DELETE /identity/v3/projects/29fe25acd5a841f7b9d864cea1c59d66 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:12:35.781250 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e308d2da-8a74-46f9-902e-ec4326301cbd None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:35.781538 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e308d2da-8a74-46f9-902e-ec4326301cbd None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:35.781817 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e308d2da-8a74-46f9-902e-ec4326301cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:35.799184 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-e308d2da-8a74-46f9-902e-ec4326301cbd None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:12:35.800410 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-e308d2da-8a74-46f9-902e-ec4326301cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:12:35.858757 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1414/2836] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:12:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:12:35.866850 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bdca73-9a20-4e87-8a75-e96cb64e62fd None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:35.867139 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bdca73-9a20-4e87-8a75-e96cb64e62fd None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:35.867419 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bdca73-9a20-4e87-8a75-e96cb64e62fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:35.886471 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-b4bdca73-9a20-4e87-8a75-e96cb64e62fd None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:12:35.887682 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-b4bdca73-9a20-4e87-8a75-e96cb64e62fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:12:35.895870 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1423/2837] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:12:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:12:35.900740 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e308d2da-8a74-46f9-902e-ec4326301cbd None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:35.910679 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e308d2da-8a74-46f9-902e-ec4326301cbd None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:35.915214 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f94dc6-b043-4724-8612-95a88111efb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:35.915991 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f94dc6-b043-4724-8612-95a88111efb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:35.916309 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f94dc6-b043-4724-8612-95a88111efb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:35.916587 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f94dc6-b043-4724-8612-95a88111efb2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:35.917300 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f94dc6-b043-4724-8612-95a88111efb2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80601) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:12:35.917605 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f94dc6-b043-4724-8612-95a88111efb2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:35.919174 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f94dc6-b043-4724-8612-95a88111efb2 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:35.930871 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1415/2838] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jun 15 15:12:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:12:35.935315 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bdca73-9a20-4e87-8a75-e96cb64e62fd None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:35.945196 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bdca73-9a20-4e87-8a75-e96cb64e62fd None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:35.950110 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2327d2b-10ce-46f1-8f2c-f0a11804bcf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:35.950788 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2327d2b-10ce-46f1-8f2c-f0a11804bcf0 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:35.951078 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2327d2b-10ce-46f1-8f2c-f0a11804bcf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:35.951357 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2327d2b-10ce-46f1-8f2c-f0a11804bcf0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:35.952045 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2327d2b-10ce-46f1-8f2c-f0a11804bcf0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:35.953967 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2327d2b-10ce-46f1-8f2c-f0a11804bcf0 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:36.034382 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1424/2839] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 15:12:35 2022] POST /identity/v3/projects => generated 379 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:12:36.038785 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f94dc6-b043-4724-8612-95a88111efb2 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:36.048761 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f94dc6-b043-4724-8612-95a88111efb2 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:36.056460 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:12:36.105013 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d99581f-36f5-4a16-b581-d3be57b42243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:36.105724 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d99581f-36f5-4a16-b581-d3be57b42243 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:36.106007 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d99581f-36f5-4a16-b581-d3be57b42243 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:36.106283 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d99581f-36f5-4a16-b581-d3be57b42243 admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:36.106972 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d99581f-36f5-4a16-b581-d3be57b42243 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:36.108987 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d99581f-36f5-4a16-b581-d3be57b42243 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:36.193151 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1416/2840] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 15:12:36 2022] POST /identity/v3/users => generated 290 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:12:36.198074 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2327d2b-10ce-46f1-8f2c-f0a11804bcf0 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:36.208591 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2327d2b-10ce-46f1-8f2c-f0a11804bcf0 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:36.213818 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f09164-77a8-41b5-ad09-c0fad58f7d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:36.214471 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f09164-77a8-41b5-ad09-c0fad58f7d9c admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:36.214882 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f09164-77a8-41b5-ad09-c0fad58f7d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:36.215174 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f09164-77a8-41b5-ad09-c0fad58f7d9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:36.215909 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f09164-77a8-41b5-ad09-c0fad58f7d9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:36.217546 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f09164-77a8-41b5-ad09-c0fad58f7d9c admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:36.228897 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1425/2841] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 15:12:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:12:36.233080 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d99581f-36f5-4a16-b581-d3be57b42243 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:36.243763 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d99581f-36f5-4a16-b581-d3be57b42243 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:36.248990 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8d69ce-e0cd-4e27-b368-1f235d74798e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:36.249640 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8d69ce-e0cd-4e27-b368-1f235d74798e admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:36.249924 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8d69ce-e0cd-4e27-b368-1f235d74798e admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:36.250231 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8d69ce-e0cd-4e27-b368-1f235d74798e admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:36.250948 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8d69ce-e0cd-4e27-b368-1f235d74798e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:36.252578 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8d69ce-e0cd-4e27-b368-1f235d74798e admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:36.262606 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1417/2842] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 15:12:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:12:36.266809 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f09164-77a8-41b5-ad09-c0fad58f7d9c admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:36.277341 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f09164-77a8-41b5-ad09-c0fad58f7d9c admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:36.281919 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf77ee78-81fb-4afc-a675-2bb9f3b28321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:36.282790 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77ee78-81fb-4afc-a675-2bb9f3b28321 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:36.283074 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77ee78-81fb-4afc-a675-2bb9f3b28321 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:36.283353 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77ee78-81fb-4afc-a675-2bb9f3b28321 admin admin] PATH_INFO: `/v3/projects/b2b3dd29981449dd8936bec9ccad1514/users/aef6decb998d4bc1af70d3df1f27b7b4/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:36.295042 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:12:36.298889 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf77ee78-81fb-4afc-a675-2bb9f3b28321 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2b3dd29981449dd8936bec9ccad1514, user_id=aef6decb998d4bc1af70d3df1f27b7b4, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:36.301549 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf77ee78-81fb-4afc-a675-2bb9f3b28321 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:36.392973 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1426/2843] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 15:12:36 2022] PUT /identity/v3/projects/b2b3dd29981449dd8936bec9ccad1514/users/aef6decb998d4bc1af70d3df1f27b7b4/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:12:36.404901 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f167c8e-5887-4fb3-9115-b99294489935 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:36.405188 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f167c8e-5887-4fb3-9115-b99294489935 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:36.405463 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f167c8e-5887-4fb3-9115-b99294489935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:36.434340 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-3f167c8e-5887-4fb3-9115-b99294489935 None None] MFA Rules not processed for user `aef6decb998d4bc1af70d3df1f27b7b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:12:36.435551 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-3f167c8e-5887-4fb3-9115-b99294489935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:12:36.499312 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1418/2844] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:12:36 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:12:36.503699 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf77ee78-81fb-4afc-a675-2bb9f3b28321 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:36.568859 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf77ee78-81fb-4afc-a675-2bb9f3b28321 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:36.574462 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f87731-7955-4148-a2dc-819c68a1d080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:36.575121 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f87731-7955-4148-a2dc-819c68a1d080 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:36.581280 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f87731-7955-4148-a2dc-819c68a1d080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:36.581280 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f87731-7955-4148-a2dc-819c68a1d080 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:36.581280 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f87731-7955-4148-a2dc-819c68a1d080 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:36.581280 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f87731-7955-4148-a2dc-819c68a1d080 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:36.653632 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1427/2845] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 15:12:36 2022] POST /identity/v3/projects => generated 377 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:12:36.656382 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f167c8e-5887-4fb3-9115-b99294489935 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:36.667431 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f167c8e-5887-4fb3-9115-b99294489935 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:36.722979 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefae7da-04f6-453f-9754-0999e8b46fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:36.723640 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefae7da-04f6-453f-9754-0999e8b46fb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:36.723922 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefae7da-04f6-453f-9754-0999e8b46fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:36.724226 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefae7da-04f6-453f-9754-0999e8b46fb4 admin admin] PATH_INFO: `/v3/users` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:36.724923 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefae7da-04f6-453f-9754-0999e8b46fb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:36.726887 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefae7da-04f6-453f-9754-0999e8b46fb4 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:36.810722 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1419/2846] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 15:12:36 2022] POST /identity/v3/users => generated 295 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:12:36.814692 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f87731-7955-4148-a2dc-819c68a1d080 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:36.825753 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f87731-7955-4148-a2dc-819c68a1d080 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:36.830370 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24caf696-bb02-4c7a-8b55-f0d2b63887e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:36.831023 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24caf696-bb02-4c7a-8b55-f0d2b63887e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:36.831332 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24caf696-bb02-4c7a-8b55-f0d2b63887e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:36.831640 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24caf696-bb02-4c7a-8b55-f0d2b63887e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:36.832443 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24caf696-bb02-4c7a-8b55-f0d2b63887e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:36.834013 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24caf696-bb02-4c7a-8b55-f0d2b63887e8 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:36.845616 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1428/2847] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 15:12:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:12:36.849761 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefae7da-04f6-453f-9754-0999e8b46fb4 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:36.860549 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefae7da-04f6-453f-9754-0999e8b46fb4 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:36.865178 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c09a23-2077-498f-ac4b-02a631905841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:36.866053 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c09a23-2077-498f-ac4b-02a631905841 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:36.866905 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c09a23-2077-498f-ac4b-02a631905841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:36.867246 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c09a23-2077-498f-ac4b-02a631905841 admin admin] PATH_INFO: `/v3/projects/84f24cb4fe574719802571304b62df5b/users/c26173ebb2bd4419a7efe912a3acf4c2/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:36.871409 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:12:36.883169 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c09a23-2077-498f-ac4b-02a631905841 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84f24cb4fe574719802571304b62df5b, user_id=c26173ebb2bd4419a7efe912a3acf4c2, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:36.885862 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c09a23-2077-498f-ac4b-02a631905841 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:36.953486 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1420/2848] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 15:12:36 2022] PUT /identity/v3/projects/84f24cb4fe574719802571304b62df5b/users/c26173ebb2bd4419a7efe912a3acf4c2/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:12:36.957603 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24caf696-bb02-4c7a-8b55-f0d2b63887e8 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:36.967554 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24caf696-bb02-4c7a-8b55-f0d2b63887e8 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:37.025090 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa19914a-a8f3-4935-bd07-73b568f0cbe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:37.025753 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa19914a-a8f3-4935-bd07-73b568f0cbe3 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:37.026036 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa19914a-a8f3-4935-bd07-73b568f0cbe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:37.026313 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa19914a-a8f3-4935-bd07-73b568f0cbe3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:37.027039 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa19914a-a8f3-4935-bd07-73b568f0cbe3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:37.028685 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa19914a-a8f3-4935-bd07-73b568f0cbe3 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:37.038866 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1429/2849] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 15:12:36 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 15:12:37.042999 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c09a23-2077-498f-ac4b-02a631905841 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:37.053124 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c09a23-2077-498f-ac4b-02a631905841 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:37.058121 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820230b1-695e-4ceb-afbd-893422f88591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:37.058981 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820230b1-695e-4ceb-afbd-893422f88591 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:37.059264 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820230b1-695e-4ceb-afbd-893422f88591 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:37.059542 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820230b1-695e-4ceb-afbd-893422f88591 admin admin] PATH_INFO: `/v3/projects/84f24cb4fe574719802571304b62df5b/users/c26173ebb2bd4419a7efe912a3acf4c2/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:37.061532 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820230b1-695e-4ceb-afbd-893422f88591 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84f24cb4fe574719802571304b62df5b, user_id=c26173ebb2bd4419a7efe912a3acf4c2, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:37.064143 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820230b1-695e-4ceb-afbd-893422f88591 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:37.074735 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1421/2850] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 15:12:37 2022] PUT /identity/v3/projects/84f24cb4fe574719802571304b62df5b/users/c26173ebb2bd4419a7efe912a3acf4c2/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:12:37.084479 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5a5502-d8ea-4699-9ea4-523258978484 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:37.084796 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5a5502-d8ea-4699-9ea4-523258978484 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:37.085908 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5a5502-d8ea-4699-9ea4-523258978484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:37.111944 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-4d5a5502-d8ea-4699-9ea4-523258978484 None None] MFA Rules not processed for user `c26173ebb2bd4419a7efe912a3acf4c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:12:37.113172 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-4d5a5502-d8ea-4699-9ea4-523258978484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:12:37.160380 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:12:37.181893 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1430/2851] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:12:37 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:12:37.193962 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820230b1-695e-4ceb-afbd-893422f88591 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:37.195817 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-820230b1-695e-4ceb-afbd-893422f88591 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:12:37.253744 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820230b1-695e-4ceb-afbd-893422f88591 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:37.258163 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4158180a-7066-4720-baff-a1ded3b9d33d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:37.258835 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4158180a-7066-4720-baff-a1ded3b9d33d service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:37.259120 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4158180a-7066-4720-baff-a1ded3b9d33d service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:37.259398 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4158180a-7066-4720-baff-a1ded3b9d33d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:37.268595 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4158180a-7066-4720-baff-a1ded3b9d33d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:37.270016 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4158180a-7066-4720-baff-a1ded3b9d33d service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:37.280325 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1422/2852] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:12:37 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:12:38.205892 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5a5502-d8ea-4699-9ea4-523258978484 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:38.215065 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5a5502-d8ea-4699-9ea4-523258978484 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:38.219954 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c79bc0-1c2e-4c6b-8e8d-1aeacbdf4727 tempest-VolumeQuotaClassesTest-74707593 tempest-VolumeQuotaClassesTest-74707593-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c26173ebb2bd4419a7efe912a3acf4c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f24cb4fe574719802571304b62df5b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:38.220628 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c79bc0-1c2e-4c6b-8e8d-1aeacbdf4727 tempest-VolumeQuotaClassesTest-74707593 tempest-VolumeQuotaClassesTest-74707593-project-admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:38.220917 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c79bc0-1c2e-4c6b-8e8d-1aeacbdf4727 tempest-VolumeQuotaClassesTest-74707593 tempest-VolumeQuotaClassesTest-74707593-project-admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:38.221197 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c79bc0-1c2e-4c6b-8e8d-1aeacbdf4727 tempest-VolumeQuotaClassesTest-74707593 tempest-VolumeQuotaClassesTest-74707593-project-admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:38.221934 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c79bc0-1c2e-4c6b-8e8d-1aeacbdf4727 tempest-VolumeQuotaClassesTest-74707593 tempest-VolumeQuotaClassesTest-74707593-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:12:38.222255 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c79bc0-1c2e-4c6b-8e8d-1aeacbdf4727 tempest-VolumeQuotaClassesTest-74707593 tempest-VolumeQuotaClassesTest-74707593-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:38.223839 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c79bc0-1c2e-4c6b-8e8d-1aeacbdf4727 tempest-VolumeQuotaClassesTest-74707593 tempest-VolumeQuotaClassesTest-74707593-project-admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:38.236347 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1431/2853] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jun 15 15:12:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:12:38.240253 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4158180a-7066-4720-baff-a1ded3b9d33d service cinder] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:38.250245 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4158180a-7066-4720-baff-a1ded3b9d33d service cinder] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:38.255055 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cf8003-4571-43cf-a6b5-93de932a53b6 tempest-VolumeQuotaClassesTest-74707593 tempest-VolumeQuotaClassesTest-74707593-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c26173ebb2bd4419a7efe912a3acf4c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f24cb4fe574719802571304b62df5b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:38.262972 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cf8003-4571-43cf-a6b5-93de932a53b6 tempest-VolumeQuotaClassesTest-74707593 tempest-VolumeQuotaClassesTest-74707593-project-admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:38.262972 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cf8003-4571-43cf-a6b5-93de932a53b6 tempest-VolumeQuotaClassesTest-74707593 tempest-VolumeQuotaClassesTest-74707593-project-admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:38.262972 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cf8003-4571-43cf-a6b5-93de932a53b6 tempest-VolumeQuotaClassesTest-74707593 tempest-VolumeQuotaClassesTest-74707593-project-admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:38.262972 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cf8003-4571-43cf-a6b5-93de932a53b6 tempest-VolumeQuotaClassesTest-74707593 tempest-VolumeQuotaClassesTest-74707593-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:38.262972 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cf8003-4571-43cf-a6b5-93de932a53b6 tempest-VolumeQuotaClassesTest-74707593 tempest-VolumeQuotaClassesTest-74707593-project-admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:38.349039 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1423/2854] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 15:12:38 2022] POST /identity/v3/projects => generated 354 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:12:38.353247 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c79bc0-1c2e-4c6b-8e8d-1aeacbdf4727 tempest-VolumeQuotaClassesTest-74707593 tempest-VolumeQuotaClassesTest-74707593-project-admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:38.363812 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c79bc0-1c2e-4c6b-8e8d-1aeacbdf4727 tempest-VolumeQuotaClassesTest-74707593 tempest-VolumeQuotaClassesTest-74707593-project-admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:38.421352 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296383df-5b1e-450b-bb91-68d469493573 tempest-VolumeQuotaClassesTest-74707593 tempest-VolumeQuotaClassesTest-74707593-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c26173ebb2bd4419a7efe912a3acf4c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f24cb4fe574719802571304b62df5b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:38.422015 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296383df-5b1e-450b-bb91-68d469493573 tempest-VolumeQuotaClassesTest-74707593 tempest-VolumeQuotaClassesTest-74707593-project-admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:38.422303 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296383df-5b1e-450b-bb91-68d469493573 tempest-VolumeQuotaClassesTest-74707593 tempest-VolumeQuotaClassesTest-74707593-project-admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:38.422581 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296383df-5b1e-450b-bb91-68d469493573 tempest-VolumeQuotaClassesTest-74707593 tempest-VolumeQuotaClassesTest-74707593-project-admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:38.423302 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296383df-5b1e-450b-bb91-68d469493573 tempest-VolumeQuotaClassesTest-74707593 tempest-VolumeQuotaClassesTest-74707593-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:12:38.423614 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296383df-5b1e-450b-bb91-68d469493573 tempest-VolumeQuotaClassesTest-74707593 tempest-VolumeQuotaClassesTest-74707593-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:38.425288 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296383df-5b1e-450b-bb91-68d469493573 tempest-VolumeQuotaClassesTest-74707593 tempest-VolumeQuotaClassesTest-74707593-project-admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:38.436755 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1432/2855] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jun 15 15:12:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:12:38.471609 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cf8003-4571-43cf-a6b5-93de932a53b6 tempest-VolumeQuotaClassesTest-74707593 tempest-VolumeQuotaClassesTest-74707593-project-admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:38.482389 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cf8003-4571-43cf-a6b5-93de932a53b6 tempest-VolumeQuotaClassesTest-74707593 tempest-VolumeQuotaClassesTest-74707593-project-admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:38.486961 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d279d116-1bac-4430-be99-446d1e7cdbf2 tempest-VolumeQuotaClassesTest-74707593 tempest-VolumeQuotaClassesTest-74707593-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c26173ebb2bd4419a7efe912a3acf4c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f24cb4fe574719802571304b62df5b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:38.487812 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d279d116-1bac-4430-be99-446d1e7cdbf2 tempest-VolumeQuotaClassesTest-74707593 tempest-VolumeQuotaClassesTest-74707593-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:38.488136 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d279d116-1bac-4430-be99-446d1e7cdbf2 tempest-VolumeQuotaClassesTest-74707593 tempest-VolumeQuotaClassesTest-74707593-project-admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:38.488426 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d279d116-1bac-4430-be99-446d1e7cdbf2 tempest-VolumeQuotaClassesTest-74707593 tempest-VolumeQuotaClassesTest-74707593-project-admin] PATH_INFO: `/v3/projects/e204cc5c9a2c455099f18831b1cf9a00` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:38.489627 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d279d116-1bac-4430-be99-446d1e7cdbf2 tempest-VolumeQuotaClassesTest-74707593 tempest-VolumeQuotaClassesTest-74707593-project-admin] RBAC: Authorizing `identity:delete_project(project_id=e204cc5c9a2c455099f18831b1cf9a00)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:38.491507 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d279d116-1bac-4430-be99-446d1e7cdbf2 tempest-VolumeQuotaClassesTest-74707593 tempest-VolumeQuotaClassesTest-74707593-project-admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:38.604382 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-d279d116-1bac-4430-be99-446d1e7cdbf2 tempest-VolumeQuotaClassesTest-74707593 tempest-VolumeQuotaClassesTest-74707593-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e204cc5c9a2c455099f18831b1cf9a00', 'request_id': 'req-d279d116-1bac-4430-be99-446d1e7cdbf2'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:38.664718 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-d279d116-1bac-4430-be99-446d1e7cdbf2 tempest-VolumeQuotaClassesTest-74707593 tempest-VolumeQuotaClassesTest-74707593-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e204cc5c9a2c455099f18831b1cf9a00', 'request_id': 'req-d279d116-1bac-4430-be99-446d1e7cdbf2'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:38.671667 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-d279d116-1bac-4430-be99-446d1e7cdbf2 tempest-VolumeQuotaClassesTest-74707593 tempest-VolumeQuotaClassesTest-74707593-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e204cc5c9a2c455099f18831b1cf9a00', 'request_id': 'req-d279d116-1bac-4430-be99-446d1e7cdbf2'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:38.683099 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-d279d116-1bac-4430-be99-446d1e7cdbf2 tempest-VolumeQuotaClassesTest-74707593 tempest-VolumeQuotaClassesTest-74707593-project-admin] The token cache is being invalidate because project e204cc5c9a2c455099f18831b1cf9a00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:12:38.683388 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-d279d116-1bac-4430-be99-446d1e7cdbf2 tempest-VolumeQuotaClassesTest-74707593 tempest-VolumeQuotaClassesTest-74707593-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:38.684040 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-d279d116-1bac-4430-be99-446d1e7cdbf2 tempest-VolumeQuotaClassesTest-74707593 tempest-VolumeQuotaClassesTest-74707593-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:38.685642 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1424/2856] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 15:12:38 2022] DELETE /identity/v3/projects/e204cc5c9a2c455099f18831b1cf9a00 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:12:39.255618 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296383df-5b1e-450b-bb91-68d469493573 tempest-VolumeQuotaClassesTest-74707593 tempest-VolumeQuotaClassesTest-74707593-project-admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:39.257387 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-296383df-5b1e-450b-bb91-68d469493573 tempest-VolumeQuotaClassesTest-74707593 tempest-VolumeQuotaClassesTest-74707593-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:12:39.320684 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296383df-5b1e-450b-bb91-68d469493573 tempest-VolumeQuotaClassesTest-74707593 tempest-VolumeQuotaClassesTest-74707593-project-admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:39.375902 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d21d821-ba30-4d53-917e-a9a28c814320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:39.376707 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d21d821-ba30-4d53-917e-a9a28c814320 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:39.376986 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d21d821-ba30-4d53-917e-a9a28c814320 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:39.377260 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d21d821-ba30-4d53-917e-a9a28c814320 admin admin] PATH_INFO: `/v3/users/aef6decb998d4bc1af70d3df1f27b7b4` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:39.378419 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d21d821-ba30-4d53-917e-a9a28c814320 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aef6decb998d4bc1af70d3df1f27b7b4)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:39.380382 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d21d821-ba30-4d53-917e-a9a28c814320 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:39.394596 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:12:39.615775 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-7d21d821-ba30-4d53-917e-a9a28c814320 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aef6decb998d4bc1af70d3df1f27b7b4'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:39.616456 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-7d21d821-ba30-4d53-917e-a9a28c814320 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aef6decb998d4bc1af70d3df1f27b7b4'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:39.617066 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-7d21d821-ba30-4d53-917e-a9a28c814320 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aef6decb998d4bc1af70d3df1f27b7b4'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:39.713170 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-7d21d821-ba30-4d53-917e-a9a28c814320 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aef6decb998d4bc1af70d3df1f27b7b4'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:39.725650 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-7d21d821-ba30-4d53-917e-a9a28c814320 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aef6decb998d4bc1af70d3df1f27b7b4'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:39.738429 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1433/2857] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 15:12:39 2022] DELETE /identity/v3/users/aef6decb998d4bc1af70d3df1f27b7b4 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:12:39.743362 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b6ae15-2f67-4614-aa16-7d94369110f4 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:39.743834 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b6ae15-2f67-4614-aa16-7d94369110f4 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:39.743834 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b6ae15-2f67-4614-aa16-7d94369110f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:39.761991 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-33b6ae15-2f67-4614-aa16-7d94369110f4 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:12:39.763191 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-33b6ae15-2f67-4614-aa16-7d94369110f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:12:39.793742 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:12:39.821503 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1425/2858] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:12:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:12:39.831206 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d21d821-ba30-4d53-917e-a9a28c814320 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:39.834763 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-7d21d821-ba30-4d53-917e-a9a28c814320 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:12:39.891590 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d21d821-ba30-4d53-917e-a9a28c814320 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:39.895118 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79faac3-c389-4174-8acf-ae5a113fac5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:39.895763 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79faac3-c389-4174-8acf-ae5a113fac5c service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:39.896048 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79faac3-c389-4174-8acf-ae5a113fac5c service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:39.896394 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79faac3-c389-4174-8acf-ae5a113fac5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:39.905291 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79faac3-c389-4174-8acf-ae5a113fac5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:39.906715 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79faac3-c389-4174-8acf-ae5a113fac5c service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:39.916107 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1434/2859] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:12:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:12:40.303256 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b6ae15-2f67-4614-aa16-7d94369110f4 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:40.304697 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-33b6ae15-2f67-4614-aa16-7d94369110f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:12:40.317107 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b6ae15-2f67-4614-aa16-7d94369110f4 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:40.322173 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f90a6d-802c-4442-83e3-b67b3184d1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:40.323233 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f90a6d-802c-4442-83e3-b67b3184d1d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:40.323514 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f90a6d-802c-4442-83e3-b67b3184d1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:40.323790 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f90a6d-802c-4442-83e3-b67b3184d1d5 admin admin] PATH_INFO: `/v3/projects/b2b3dd29981449dd8936bec9ccad1514` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:40.325000 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f90a6d-802c-4442-83e3-b67b3184d1d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2b3dd29981449dd8936bec9ccad1514)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:40.326913 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f90a6d-802c-4442-83e3-b67b3184d1d5 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:40.416667 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-c3f90a6d-802c-4442-83e3-b67b3184d1d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2b3dd29981449dd8936bec9ccad1514', 'request_id': 'req-c3f90a6d-802c-4442-83e3-b67b3184d1d5'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:40.461435 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-c3f90a6d-802c-4442-83e3-b67b3184d1d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2b3dd29981449dd8936bec9ccad1514', 'request_id': 'req-c3f90a6d-802c-4442-83e3-b67b3184d1d5'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:40.467232 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-c3f90a6d-802c-4442-83e3-b67b3184d1d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2b3dd29981449dd8936bec9ccad1514', 'request_id': 'req-c3f90a6d-802c-4442-83e3-b67b3184d1d5'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:40.478117 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-c3f90a6d-802c-4442-83e3-b67b3184d1d5 admin admin] The token cache is being invalidate because project b2b3dd29981449dd8936bec9ccad1514 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:12:40.478897 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-c3f90a6d-802c-4442-83e3-b67b3184d1d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:40.479528 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-c3f90a6d-802c-4442-83e3-b67b3184d1d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:40.481087 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1426/2860] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 15:12:40 2022] DELETE /identity/v3/projects/b2b3dd29981449dd8936bec9ccad1514 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:12:40.485043 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79faac3-c389-4174-8acf-ae5a113fac5c service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:40.496235 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79faac3-c389-4174-8acf-ae5a113fac5c service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:40.555250 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915beb2c-9b66-4404-949f-4c17174d097d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:40.556171 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915beb2c-9b66-4404-949f-4c17174d097d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:40.556967 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915beb2c-9b66-4404-949f-4c17174d097d admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:40.557358 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915beb2c-9b66-4404-949f-4c17174d097d admin admin] PATH_INFO: `/v3/users/c26173ebb2bd4419a7efe912a3acf4c2` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:40.558933 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915beb2c-9b66-4404-949f-4c17174d097d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c26173ebb2bd4419a7efe912a3acf4c2)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:40.561174 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915beb2c-9b66-4404-949f-4c17174d097d admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:40.744598 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:12:40.753384 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-915beb2c-9b66-4404-949f-4c17174d097d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c26173ebb2bd4419a7efe912a3acf4c2'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:40.754037 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-915beb2c-9b66-4404-949f-4c17174d097d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c26173ebb2bd4419a7efe912a3acf4c2'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:40.754643 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-915beb2c-9b66-4404-949f-4c17174d097d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c26173ebb2bd4419a7efe912a3acf4c2'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:40.825322 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-915beb2c-9b66-4404-949f-4c17174d097d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c26173ebb2bd4419a7efe912a3acf4c2'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:40.837853 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-915beb2c-9b66-4404-949f-4c17174d097d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c26173ebb2bd4419a7efe912a3acf4c2'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:40.850576 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1435/2861] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 15:12:40 2022] DELETE /identity/v3/users/c26173ebb2bd4419a7efe912a3acf4c2 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:12:41.223262 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f90a6d-802c-4442-83e3-b67b3184d1d5 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:41.226577 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-c3f90a6d-802c-4442-83e3-b67b3184d1d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:12:41.238594 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f90a6d-802c-4442-83e3-b67b3184d1d5 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:41.295583 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6de899-8824-40cc-add0-a416a2ed20cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:41.296477 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6de899-8824-40cc-add0-a416a2ed20cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:41.296765 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6de899-8824-40cc-add0-a416a2ed20cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:41.297044 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6de899-8824-40cc-add0-a416a2ed20cc admin admin] PATH_INFO: `/v3/projects/84f24cb4fe574719802571304b62df5b` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:41.298212 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6de899-8824-40cc-add0-a416a2ed20cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=84f24cb4fe574719802571304b62df5b)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:41.300119 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6de899-8824-40cc-add0-a416a2ed20cc admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:41.421923 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-8b6de899-8824-40cc-add0-a416a2ed20cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84f24cb4fe574719802571304b62df5b', 'request_id': 'req-8b6de899-8824-40cc-add0-a416a2ed20cc'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:41.496212 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-8b6de899-8824-40cc-add0-a416a2ed20cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84f24cb4fe574719802571304b62df5b', 'request_id': 'req-8b6de899-8824-40cc-add0-a416a2ed20cc'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:41.500371 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:12:41.502774 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-8b6de899-8824-40cc-add0-a416a2ed20cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84f24cb4fe574719802571304b62df5b', 'request_id': 'req-8b6de899-8824-40cc-add0-a416a2ed20cc'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:41.515013 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-8b6de899-8824-40cc-add0-a416a2ed20cc admin admin] The token cache is being invalidate because project 84f24cb4fe574719802571304b62df5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:12:41.515304 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-8b6de899-8824-40cc-add0-a416a2ed20cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:41.515954 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-8b6de899-8824-40cc-add0-a416a2ed20cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:41.517558 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1427/2862] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 15:12:41 2022] DELETE /identity/v3/projects/84f24cb4fe574719802571304b62df5b => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:12:41.522516 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5020d56-824d-4454-bc49-5961598d59c3 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:41.522999 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5020d56-824d-4454-bc49-5961598d59c3 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:41.523105 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5020d56-824d-4454-bc49-5961598d59c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:41.544152 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-a5020d56-824d-4454-bc49-5961598d59c3 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:12:41.545358 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a5020d56-824d-4454-bc49-5961598d59c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:12:41.607368 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1436/2863] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:12:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:12:41.615057 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dfd5f0-a8d7-43df-8f72-ac62bb925f0a None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:41.615344 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dfd5f0-a8d7-43df-8f72-ac62bb925f0a None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:41.615629 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dfd5f0-a8d7-43df-8f72-ac62bb925f0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:41.634304 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-d6dfd5f0-a8d7-43df-8f72-ac62bb925f0a None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:12:41.635494 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-d6dfd5f0-a8d7-43df-8f72-ac62bb925f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:12:41.643907 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1428/2864] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:12:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:12:41.648044 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5020d56-824d-4454-bc49-5961598d59c3 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:41.657772 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5020d56-824d-4454-bc49-5961598d59c3 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:41.662304 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c20802d-f57f-407d-a4dc-b0679f4febe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:41.662946 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c20802d-f57f-407d-a4dc-b0679f4febe3 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:41.663227 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c20802d-f57f-407d-a4dc-b0679f4febe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:41.663503 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c20802d-f57f-407d-a4dc-b0679f4febe3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:41.664260 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c20802d-f57f-407d-a4dc-b0679f4febe3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:12:41.664565 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c20802d-f57f-407d-a4dc-b0679f4febe3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:41.666122 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c20802d-f57f-407d-a4dc-b0679f4febe3 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:41.678108 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1437/2865] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jun 15 15:12:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:12:41.682271 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dfd5f0-a8d7-43df-8f72-ac62bb925f0a None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:41.691970 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dfd5f0-a8d7-43df-8f72-ac62bb925f0a None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:41.696589 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097dcf44-52b1-4ef2-8c7b-e5b4282e0454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:41.697230 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097dcf44-52b1-4ef2-8c7b-e5b4282e0454 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:41.697517 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097dcf44-52b1-4ef2-8c7b-e5b4282e0454 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:41.697787 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097dcf44-52b1-4ef2-8c7b-e5b4282e0454 admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:41.699079 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097dcf44-52b1-4ef2-8c7b-e5b4282e0454 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:41.700975 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097dcf44-52b1-4ef2-8c7b-e5b4282e0454 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:41.800869 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1429/2866] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 15:12:41 2022] POST /identity/v3/projects => generated 391 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:12:41.804954 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c20802d-f57f-407d-a4dc-b0679f4febe3 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:41.814909 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c20802d-f57f-407d-a4dc-b0679f4febe3 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:41.871872 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91271e8-462e-493c-a33f-f0d2a686d53b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:41.872551 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91271e8-462e-493c-a33f-f0d2a686d53b admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:41.872833 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91271e8-462e-493c-a33f-f0d2a686d53b admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:41.873109 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91271e8-462e-493c-a33f-f0d2a686d53b admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:41.873785 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91271e8-462e-493c-a33f-f0d2a686d53b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:41.875684 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91271e8-462e-493c-a33f-f0d2a686d53b admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:41.954995 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1438/2867] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 15:12:41 2022] POST /identity/v3/users => generated 296 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:12:41.959131 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097dcf44-52b1-4ef2-8c7b-e5b4282e0454 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:41.970683 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097dcf44-52b1-4ef2-8c7b-e5b4282e0454 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:41.975356 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c049f72-2058-4bd8-95f6-66719347858f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:41.976204 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c049f72-2058-4bd8-95f6-66719347858f admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:41.976499 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c049f72-2058-4bd8-95f6-66719347858f admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:41.976780 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c049f72-2058-4bd8-95f6-66719347858f admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:41.977507 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c049f72-2058-4bd8-95f6-66719347858f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:41.979167 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c049f72-2058-4bd8-95f6-66719347858f admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:41.990038 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1430/2868] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 15:12:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:12:41.994267 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91271e8-462e-493c-a33f-f0d2a686d53b admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:42.004808 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91271e8-462e-493c-a33f-f0d2a686d53b admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:42.009618 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71997288-3967-46a3-99e1-7739568e127c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:42.010272 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71997288-3967-46a3-99e1-7739568e127c admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:42.010554 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71997288-3967-46a3-99e1-7739568e127c admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:42.010832 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71997288-3967-46a3-99e1-7739568e127c admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:42.012201 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71997288-3967-46a3-99e1-7739568e127c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:42.013771 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71997288-3967-46a3-99e1-7739568e127c admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:42.026527 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1439/2869] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 15:12:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 15:12:42.028861 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c049f72-2058-4bd8-95f6-66719347858f admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:42.039254 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c049f72-2058-4bd8-95f6-66719347858f admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:42.044025 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eb7422-1d58-4f4a-8d8f-dadb5fdb4dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:42.044927 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eb7422-1d58-4f4a-8d8f-dadb5fdb4dc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:42.045333 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eb7422-1d58-4f4a-8d8f-dadb5fdb4dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:42.045608 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eb7422-1d58-4f4a-8d8f-dadb5fdb4dc0 admin admin] PATH_INFO: `/v3/projects/237bb843ed4a4b7cbeb67668936e3ecc/users/c07a679fceab4eaf815c5df7058e299d/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:42.060794 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88eb7422-1d58-4f4a-8d8f-dadb5fdb4dc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=237bb843ed4a4b7cbeb67668936e3ecc, user_id=c07a679fceab4eaf815c5df7058e299d, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:42.063406 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88eb7422-1d58-4f4a-8d8f-dadb5fdb4dc0 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:42.144858 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1431/2870] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 15:12:42 2022] PUT /identity/v3/projects/237bb843ed4a4b7cbeb67668936e3ecc/users/c07a679fceab4eaf815c5df7058e299d/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:12:42.204029 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6735c0-67fb-4c9b-95d9-9d74cd144b8c None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:42.204347 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6735c0-67fb-4c9b-95d9-9d74cd144b8c None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:42.204626 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6735c0-67fb-4c9b-95d9-9d74cd144b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:42.231744 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-9d6735c0-67fb-4c9b-95d9-9d74cd144b8c None None] MFA Rules not processed for user `c07a679fceab4eaf815c5df7058e299d`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:12:42.232929 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-9d6735c0-67fb-4c9b-95d9-9d74cd144b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:12:42.241255 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:12:42.297440 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1440/2871] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:12:42 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:12:42.302070 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eb7422-1d58-4f4a-8d8f-dadb5fdb4dc0 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:42.365641 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eb7422-1d58-4f4a-8d8f-dadb5fdb4dc0 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:42.370833 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb90c544-7ddd-4ce0-b6df-3011976776b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:42.371486 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb90c544-7ddd-4ce0-b6df-3011976776b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:42.371771 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb90c544-7ddd-4ce0-b6df-3011976776b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:42.372051 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb90c544-7ddd-4ce0-b6df-3011976776b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:42.372808 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb90c544-7ddd-4ce0-b6df-3011976776b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:42.374687 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb90c544-7ddd-4ce0-b6df-3011976776b3 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:42.451994 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1432/2872] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 15:12:42 2022] POST /identity/v3/projects => generated 393 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:12:42.456147 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6735c0-67fb-4c9b-95d9-9d74cd144b8c None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:42.467388 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6735c0-67fb-4c9b-95d9-9d74cd144b8c None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:42.522684 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e332931-3b18-4f7c-a340-07388391f204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:42.523347 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e332931-3b18-4f7c-a340-07388391f204 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:42.527369 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e332931-3b18-4f7c-a340-07388391f204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:42.527369 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e332931-3b18-4f7c-a340-07388391f204 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:42.527369 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e332931-3b18-4f7c-a340-07388391f204 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:42.527369 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e332931-3b18-4f7c-a340-07388391f204 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:42.627803 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1441/2873] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 15:12:42 2022] POST /identity/v3/users => generated 303 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:12:42.631876 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb90c544-7ddd-4ce0-b6df-3011976776b3 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:42.642995 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb90c544-7ddd-4ce0-b6df-3011976776b3 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:42.647678 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901f4c0e-8b29-4a7a-be54-8f849584bbc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:42.648377 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901f4c0e-8b29-4a7a-be54-8f849584bbc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:42.648662 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901f4c0e-8b29-4a7a-be54-8f849584bbc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:42.648981 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901f4c0e-8b29-4a7a-be54-8f849584bbc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:42.649666 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901f4c0e-8b29-4a7a-be54-8f849584bbc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:42.651244 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901f4c0e-8b29-4a7a-be54-8f849584bbc5 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:42.662719 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1433/2874] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 15:12:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:12:42.666888 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e332931-3b18-4f7c-a340-07388391f204 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:42.677263 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e332931-3b18-4f7c-a340-07388391f204 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:42.681881 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0613f8c-19e7-45d9-a035-bf51008444e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:42.682759 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0613f8c-19e7-45d9-a035-bf51008444e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:42.683312 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0613f8c-19e7-45d9-a035-bf51008444e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:42.683507 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0613f8c-19e7-45d9-a035-bf51008444e5 admin admin] PATH_INFO: `/v3/projects/448b598ad9684d6aac640dd997f7f9d7/users/0451a61d17e84320a69c9f7209e8f39b/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:42.698023 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0613f8c-19e7-45d9-a035-bf51008444e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=448b598ad9684d6aac640dd997f7f9d7, user_id=0451a61d17e84320a69c9f7209e8f39b, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:42.701476 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0613f8c-19e7-45d9-a035-bf51008444e5 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:42.759037 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1442/2875] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 15:12:42 2022] PUT /identity/v3/projects/448b598ad9684d6aac640dd997f7f9d7/users/0451a61d17e84320a69c9f7209e8f39b/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:12:42.763871 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901f4c0e-8b29-4a7a-be54-8f849584bbc5 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:42.773902 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901f4c0e-8b29-4a7a-be54-8f849584bbc5 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:42.831303 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a002c5b6-7906-415a-92f0-79e88c14ca61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:42.832232 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a002c5b6-7906-415a-92f0-79e88c14ca61 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:42.832642 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a002c5b6-7906-415a-92f0-79e88c14ca61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:42.833015 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a002c5b6-7906-415a-92f0-79e88c14ca61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:42.833973 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a002c5b6-7906-415a-92f0-79e88c14ca61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:42.835541 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a002c5b6-7906-415a-92f0-79e88c14ca61 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:42.836788 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:12:42.845919 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1434/2876] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 15:12:42 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:12:42.850135 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0613f8c-19e7-45d9-a035-bf51008444e5 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:42.860636 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0613f8c-19e7-45d9-a035-bf51008444e5 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:42.865248 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00bbc86-4e59-4f06-b77d-ebbc716b37e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:42.866113 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00bbc86-4e59-4f06-b77d-ebbc716b37e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:42.866392 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00bbc86-4e59-4f06-b77d-ebbc716b37e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:42.866660 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00bbc86-4e59-4f06-b77d-ebbc716b37e3 admin admin] PATH_INFO: `/v3/projects/448b598ad9684d6aac640dd997f7f9d7/users/0451a61d17e84320a69c9f7209e8f39b/roles/2bd1979f53bb4853a50dd5bee6aa0782` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:42.868826 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00bbc86-4e59-4f06-b77d-ebbc716b37e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=448b598ad9684d6aac640dd997f7f9d7, user_id=0451a61d17e84320a69c9f7209e8f39b, role_id=2bd1979f53bb4853a50dd5bee6aa0782)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:42.872061 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00bbc86-4e59-4f06-b77d-ebbc716b37e3 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:42.882345 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1443/2877] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 15:12:42 2022] PUT /identity/v3/projects/448b598ad9684d6aac640dd997f7f9d7/users/0451a61d17e84320a69c9f7209e8f39b/roles/2bd1979f53bb4853a50dd5bee6aa0782 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:12:42.893325 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2fb9d5-0252-4e68-a555-78541c02b3b8 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:42.893847 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2fb9d5-0252-4e68-a555-78541c02b3b8 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:42.894307 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2fb9d5-0252-4e68-a555-78541c02b3b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:42.920560 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-dd2fb9d5-0252-4e68-a555-78541c02b3b8 None None] MFA Rules not processed for user `0451a61d17e84320a69c9f7209e8f39b`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:12:42.921767 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-dd2fb9d5-0252-4e68-a555-78541c02b3b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:12:42.988466 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1435/2878] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:12:42 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:12:42.999858 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00bbc86-4e59-4f06-b77d-ebbc716b37e3 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:43.001640 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-b00bbc86-4e59-4f06-b77d-ebbc716b37e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:12:43.058861 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00bbc86-4e59-4f06-b77d-ebbc716b37e3 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:43.062480 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460e12cc-adf7-4358-a732-87360f148a3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:43.063117 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e12cc-adf7-4358-a732-87360f148a3f service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:43.063402 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e12cc-adf7-4358-a732-87360f148a3f service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:43.063680 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e12cc-adf7-4358-a732-87360f148a3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:43.072703 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460e12cc-adf7-4358-a732-87360f148a3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:43.074703 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460e12cc-adf7-4358-a732-87360f148a3f service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:43.084078 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1444/2879] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:12:42 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:12:43.130949 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2fb9d5-0252-4e68-a555-78541c02b3b8 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:43.132354 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-dd2fb9d5-0252-4e68-a555-78541c02b3b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:12:43.144499 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2fb9d5-0252-4e68-a555-78541c02b3b8 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:43.148207 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b0cfb8-cc42-491f-bba2-b7b4919045f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:43.148845 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b0cfb8-cc42-491f-bba2-b7b4919045f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:43.149127 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b0cfb8-cc42-491f-bba2-b7b4919045f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:43.149402 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b0cfb8-cc42-491f-bba2-b7b4919045f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:43.208293 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b0cfb8-cc42-491f-bba2-b7b4919045f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:43.209716 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b0cfb8-cc42-491f-bba2-b7b4919045f7 service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:43.218565 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1436/2880] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:12:43 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:12:47.864624 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460e12cc-adf7-4358-a732-87360f148a3f service cinder] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:47.870985 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:12:47.928682 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460e12cc-adf7-4358-a732-87360f148a3f service cinder] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:47.933210 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fc8f45-ccd9-4fb6-ac66-76382c30a867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:47.934676 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fc8f45-ccd9-4fb6-ac66-76382c30a867 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:47.935036 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fc8f45-ccd9-4fb6-ac66-76382c30a867 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:47.935340 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fc8f45-ccd9-4fb6-ac66-76382c30a867 admin admin] PATH_INFO: `/v3/users/c07a679fceab4eaf815c5df7058e299d` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:47.936549 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fc8f45-ccd9-4fb6-ac66-76382c30a867 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c07a679fceab4eaf815c5df7058e299d)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:47.938473 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fc8f45-ccd9-4fb6-ac66-76382c30a867 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:48.123866 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-80fc8f45-ccd9-4fb6-ac66-76382c30a867 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c07a679fceab4eaf815c5df7058e299d'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:48.124506 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-80fc8f45-ccd9-4fb6-ac66-76382c30a867 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c07a679fceab4eaf815c5df7058e299d'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:48.125110 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-80fc8f45-ccd9-4fb6-ac66-76382c30a867 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c07a679fceab4eaf815c5df7058e299d'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:48.213146 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-80fc8f45-ccd9-4fb6-ac66-76382c30a867 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c07a679fceab4eaf815c5df7058e299d'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:48.226111 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-80fc8f45-ccd9-4fb6-ac66-76382c30a867 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c07a679fceab4eaf815c5df7058e299d'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:48.238828 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1445/2881] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 15:12:47 2022] DELETE /identity/v3/users/c07a679fceab4eaf815c5df7058e299d => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:12:48.243662 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17232fa4-019a-47bc-adb7-a59d64fac8f1 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:48.244214 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17232fa4-019a-47bc-adb7-a59d64fac8f1 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:48.244359 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17232fa4-019a-47bc-adb7-a59d64fac8f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:48.261500 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-17232fa4-019a-47bc-adb7-a59d64fac8f1 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:12:48.262972 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-17232fa4-019a-47bc-adb7-a59d64fac8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:12:48.324740 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1437/2882] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:12:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:12:48.334370 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fc8f45-ccd9-4fb6-ac66-76382c30a867 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:48.335774 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-80fc8f45-ccd9-4fb6-ac66-76382c30a867 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:12:48.404915 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fc8f45-ccd9-4fb6-ac66-76382c30a867 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:48.408558 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7da5e21-3213-48f5-982b-fad6a62f640f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:48.409206 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7da5e21-3213-48f5-982b-fad6a62f640f service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:48.409490 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7da5e21-3213-48f5-982b-fad6a62f640f service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:48.409768 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7da5e21-3213-48f5-982b-fad6a62f640f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:48.419066 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7da5e21-3213-48f5-982b-fad6a62f640f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:48.420677 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7da5e21-3213-48f5-982b-fad6a62f640f service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:48.430301 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1446/2883] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:12:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:12:48.846783 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17232fa4-019a-47bc-adb7-a59d64fac8f1 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:48.853014 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-17232fa4-019a-47bc-adb7-a59d64fac8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:12:48.862054 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17232fa4-019a-47bc-adb7-a59d64fac8f1 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:48.867158 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563c3639-b11e-44fe-b16b-b651abbcaa3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:48.868314 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c3639-b11e-44fe-b16b-b651abbcaa3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:48.868628 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c3639-b11e-44fe-b16b-b651abbcaa3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:48.868913 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c3639-b11e-44fe-b16b-b651abbcaa3d admin admin] PATH_INFO: `/v3/projects/237bb843ed4a4b7cbeb67668936e3ecc` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:48.870103 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563c3639-b11e-44fe-b16b-b651abbcaa3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=237bb843ed4a4b7cbeb67668936e3ecc)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:48.873174 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563c3639-b11e-44fe-b16b-b651abbcaa3d admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:48.890518 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=100 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:12:48.972650 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-563c3639-b11e-44fe-b16b-b651abbcaa3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '237bb843ed4a4b7cbeb67668936e3ecc', 'request_id': 'req-563c3639-b11e-44fe-b16b-b651abbcaa3d'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:49.038414 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-563c3639-b11e-44fe-b16b-b651abbcaa3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '237bb843ed4a4b7cbeb67668936e3ecc', 'request_id': 'req-563c3639-b11e-44fe-b16b-b651abbcaa3d'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:49.044433 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-563c3639-b11e-44fe-b16b-b651abbcaa3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '237bb843ed4a4b7cbeb67668936e3ecc', 'request_id': 'req-563c3639-b11e-44fe-b16b-b651abbcaa3d'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:49.056336 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-563c3639-b11e-44fe-b16b-b651abbcaa3d admin admin] The token cache is being invalidate because project 237bb843ed4a4b7cbeb67668936e3ecc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:12:49.056623 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-563c3639-b11e-44fe-b16b-b651abbcaa3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:49.057255 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-563c3639-b11e-44fe-b16b-b651abbcaa3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:49.059330 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1438/2884] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 15:12:48 2022] DELETE /identity/v3/projects/237bb843ed4a4b7cbeb67668936e3ecc => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:12:49.063835 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7da5e21-3213-48f5-982b-fad6a62f640f service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:49.073746 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7da5e21-3213-48f5-982b-fad6a62f640f service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:49.102690 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:12:49.135871 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e949d0c-37ec-4cd7-8bde-5592f9a5ba04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:49.136794 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e949d0c-37ec-4cd7-8bde-5592f9a5ba04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:49.137275 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e949d0c-37ec-4cd7-8bde-5592f9a5ba04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:49.137610 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e949d0c-37ec-4cd7-8bde-5592f9a5ba04 admin admin] PATH_INFO: `/v3/users/0451a61d17e84320a69c9f7209e8f39b` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:49.138975 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e949d0c-37ec-4cd7-8bde-5592f9a5ba04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0451a61d17e84320a69c9f7209e8f39b)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:49.141058 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e949d0c-37ec-4cd7-8bde-5592f9a5ba04 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:49.292148 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-6e949d0c-37ec-4cd7-8bde-5592f9a5ba04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0451a61d17e84320a69c9f7209e8f39b'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:49.292803 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-6e949d0c-37ec-4cd7-8bde-5592f9a5ba04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0451a61d17e84320a69c9f7209e8f39b'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:49.294053 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-6e949d0c-37ec-4cd7-8bde-5592f9a5ba04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0451a61d17e84320a69c9f7209e8f39b'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:49.369863 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-6e949d0c-37ec-4cd7-8bde-5592f9a5ba04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0451a61d17e84320a69c9f7209e8f39b'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:49.383801 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-6e949d0c-37ec-4cd7-8bde-5592f9a5ba04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0451a61d17e84320a69c9f7209e8f39b'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:49.396884 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1447/2885] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 15:12:49 2022] DELETE /identity/v3/users/0451a61d17e84320a69c9f7209e8f39b => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:12:49.804743 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563c3639-b11e-44fe-b16b-b651abbcaa3d admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:49.805789 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-563c3639-b11e-44fe-b16b-b651abbcaa3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:12:49.818986 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563c3639-b11e-44fe-b16b-b651abbcaa3d admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:49.878015 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d778a2-655e-4f62-a4b3-aa8c17a012da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:49.878903 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d778a2-655e-4f62-a4b3-aa8c17a012da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:49.879184 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d778a2-655e-4f62-a4b3-aa8c17a012da admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:49.879456 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d778a2-655e-4f62-a4b3-aa8c17a012da admin admin] PATH_INFO: `/v3/projects/448b598ad9684d6aac640dd997f7f9d7` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:49.880654 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d778a2-655e-4f62-a4b3-aa8c17a012da admin admin] RBAC: Authorizing `identity:delete_project(project_id=448b598ad9684d6aac640dd997f7f9d7)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:49.882603 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d778a2-655e-4f62-a4b3-aa8c17a012da admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:49.991057 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-74d778a2-655e-4f62-a4b3-aa8c17a012da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '448b598ad9684d6aac640dd997f7f9d7', 'request_id': 'req-74d778a2-655e-4f62-a4b3-aa8c17a012da'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:50.087367 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-74d778a2-655e-4f62-a4b3-aa8c17a012da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '448b598ad9684d6aac640dd997f7f9d7', 'request_id': 'req-74d778a2-655e-4f62-a4b3-aa8c17a012da'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:50.094395 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-74d778a2-655e-4f62-a4b3-aa8c17a012da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '448b598ad9684d6aac640dd997f7f9d7', 'request_id': 'req-74d778a2-655e-4f62-a4b3-aa8c17a012da'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:50.105425 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-74d778a2-655e-4f62-a4b3-aa8c17a012da admin admin] The token cache is being invalidate because project 448b598ad9684d6aac640dd997f7f9d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:12:50.105849 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-74d778a2-655e-4f62-a4b3-aa8c17a012da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:50.106630 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-74d778a2-655e-4f62-a4b3-aa8c17a012da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:50.108214 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1439/2886] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 15:12:49 2022] DELETE /identity/v3/projects/448b598ad9684d6aac640dd997f7f9d7 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:12:50.113141 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f8baf9-89bc-41cc-8c49-bda6ac79c6a9 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:50.113647 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f8baf9-89bc-41cc-8c49-bda6ac79c6a9 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:50.113791 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f8baf9-89bc-41cc-8c49-bda6ac79c6a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:50.131589 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-90f8baf9-89bc-41cc-8c49-bda6ac79c6a9 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:12:50.132837 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-90f8baf9-89bc-41cc-8c49-bda6ac79c6a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:12:50.192957 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1448/2887] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:12:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:12:50.201726 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e16af3-c2c1-4550-baf7-9f78650eb774 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:50.202269 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e16af3-c2c1-4550-baf7-9f78650eb774 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:50.202606 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e16af3-c2c1-4550-baf7-9f78650eb774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:50.221564 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-c4e16af3-c2c1-4550-baf7-9f78650eb774 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:12:50.222746 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-c4e16af3-c2c1-4550-baf7-9f78650eb774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:12:50.230467 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1440/2888] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:12:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:12:50.235065 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f8baf9-89bc-41cc-8c49-bda6ac79c6a9 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:50.245099 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f8baf9-89bc-41cc-8c49-bda6ac79c6a9 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:50.250257 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc17674-aaa6-4bc1-920a-500d63278202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:50.250910 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc17674-aaa6-4bc1-920a-500d63278202 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:50.251196 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc17674-aaa6-4bc1-920a-500d63278202 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:50.251474 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc17674-aaa6-4bc1-920a-500d63278202 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:50.252241 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc17674-aaa6-4bc1-920a-500d63278202 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:12:50.252557 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc17674-aaa6-4bc1-920a-500d63278202 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:50.254501 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc17674-aaa6-4bc1-920a-500d63278202 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:50.267172 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1449/2889] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jun 15 15:12:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:12:50.271434 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e16af3-c2c1-4550-baf7-9f78650eb774 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:50.280696 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e16af3-c2c1-4550-baf7-9f78650eb774 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:50.285204 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b31859c-89ba-4b96-84a1-58723c4c4a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:50.285842 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b31859c-89ba-4b96-84a1-58723c4c4a3c admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:50.286121 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b31859c-89ba-4b96-84a1-58723c4c4a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:50.286393 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b31859c-89ba-4b96-84a1-58723c4c4a3c admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:50.287069 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b31859c-89ba-4b96-84a1-58723c4c4a3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:50.288978 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b31859c-89ba-4b96-84a1-58723c4c4a3c admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:50.369213 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1441/2890] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 15:12:50 2022] POST /identity/v3/projects => generated 373 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:12:50.373482 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc17674-aaa6-4bc1-920a-500d63278202 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:50.383747 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc17674-aaa6-4bc1-920a-500d63278202 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:50.413688 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:12:50.439679 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad85501c-7fdc-4398-b226-276d9f5a24f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:50.440381 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad85501c-7fdc-4398-b226-276d9f5a24f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:50.440668 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad85501c-7fdc-4398-b226-276d9f5a24f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:50.440944 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad85501c-7fdc-4398-b226-276d9f5a24f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:50.441664 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad85501c-7fdc-4398-b226-276d9f5a24f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:50.443702 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad85501c-7fdc-4398-b226-276d9f5a24f3 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:50.579090 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1450/2891] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 15:12:50 2022] POST /identity/v3/users => generated 287 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:12:50.583176 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b31859c-89ba-4b96-84a1-58723c4c4a3c admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:50.589597 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:12:50.594521 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b31859c-89ba-4b96-84a1-58723c4c4a3c admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:50.599194 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3e53ec-f689-4c69-8aa4-983cdb23a1b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:50.602263 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e53ec-f689-4c69-8aa4-983cdb23a1b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:50.602263 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e53ec-f689-4c69-8aa4-983cdb23a1b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:50.602263 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e53ec-f689-4c69-8aa4-983cdb23a1b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:50.602263 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3e53ec-f689-4c69-8aa4-983cdb23a1b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:50.602907 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3e53ec-f689-4c69-8aa4-983cdb23a1b2 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:50.614268 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1442/2892] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 15:12:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:12:50.618428 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad85501c-7fdc-4398-b226-276d9f5a24f3 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:50.628948 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad85501c-7fdc-4398-b226-276d9f5a24f3 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:50.633486 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4209dd27-aa3f-4af9-8ac9-5f5b911f7f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:50.634206 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4209dd27-aa3f-4af9-8ac9-5f5b911f7f52 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:50.634468 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4209dd27-aa3f-4af9-8ac9-5f5b911f7f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:50.634760 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4209dd27-aa3f-4af9-8ac9-5f5b911f7f52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:50.635485 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4209dd27-aa3f-4af9-8ac9-5f5b911f7f52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:50.637689 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4209dd27-aa3f-4af9-8ac9-5f5b911f7f52 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:50.648131 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1451/2893] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 15:12:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:12:50.652179 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3e53ec-f689-4c69-8aa4-983cdb23a1b2 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:50.662539 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3e53ec-f689-4c69-8aa4-983cdb23a1b2 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:50.666987 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52459ce9-858c-4223-aa30-d509045c602e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:50.667935 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52459ce9-858c-4223-aa30-d509045c602e admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:50.668225 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52459ce9-858c-4223-aa30-d509045c602e admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:50.668508 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52459ce9-858c-4223-aa30-d509045c602e admin admin] PATH_INFO: `/v3/projects/9b4f2c2c197441b5ab316ffe1037de39/users/e8903b338e5b4fc9b123c4f3696f1f33/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:50.682684 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52459ce9-858c-4223-aa30-d509045c602e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b4f2c2c197441b5ab316ffe1037de39, user_id=e8903b338e5b4fc9b123c4f3696f1f33, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:50.685747 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52459ce9-858c-4223-aa30-d509045c602e admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:50.779826 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1443/2894] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 15:12:50 2022] PUT /identity/v3/projects/9b4f2c2c197441b5ab316ffe1037de39/users/e8903b338e5b4fc9b123c4f3696f1f33/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:12:50.791509 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fe56e5-68ad-437e-a6df-abe2438ccdab None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:50.791802 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fe56e5-68ad-437e-a6df-abe2438ccdab None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:50.792111 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fe56e5-68ad-437e-a6df-abe2438ccdab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:50.818915 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-91fe56e5-68ad-437e-a6df-abe2438ccdab None None] MFA Rules not processed for user `e8903b338e5b4fc9b123c4f3696f1f33`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:12:50.820150 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-91fe56e5-68ad-437e-a6df-abe2438ccdab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:12:50.884191 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1452/2895] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:12:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:12:50.896014 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52459ce9-858c-4223-aa30-d509045c602e admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:50.897787 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-52459ce9-858c-4223-aa30-d509045c602e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:12:50.955852 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52459ce9-858c-4223-aa30-d509045c602e admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:50.959449 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96865c9e-e8e3-4f10-8dd1-25b8e26f101c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:50.960097 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96865c9e-e8e3-4f10-8dd1-25b8e26f101c service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:50.960395 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96865c9e-e8e3-4f10-8dd1-25b8e26f101c service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:50.960672 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96865c9e-e8e3-4f10-8dd1-25b8e26f101c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:50.969829 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96865c9e-e8e3-4f10-8dd1-25b8e26f101c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:50.971244 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96865c9e-e8e3-4f10-8dd1-25b8e26f101c service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:50.979951 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1444/2896] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:12:50 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:12:51.000280 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fe56e5-68ad-437e-a6df-abe2438ccdab None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:51.063441 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fe56e5-68ad-437e-a6df-abe2438ccdab None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:51.068070 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231dff07-6de6-44f8-aa72-99ad6ef85e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:51.068969 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231dff07-6de6-44f8-aa72-99ad6ef85e93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:51.069274 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231dff07-6de6-44f8-aa72-99ad6ef85e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:51.069555 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231dff07-6de6-44f8-aa72-99ad6ef85e93 admin admin] PATH_INFO: `/v3/users/e8903b338e5b4fc9b123c4f3696f1f33` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:51.070757 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231dff07-6de6-44f8-aa72-99ad6ef85e93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8903b338e5b4fc9b123c4f3696f1f33)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:51.072700 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231dff07-6de6-44f8-aa72-99ad6ef85e93 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:51.259176 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:12:51.261899 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-231dff07-6de6-44f8-aa72-99ad6ef85e93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8903b338e5b4fc9b123c4f3696f1f33'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:51.262574 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-231dff07-6de6-44f8-aa72-99ad6ef85e93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8903b338e5b4fc9b123c4f3696f1f33'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:51.263865 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-231dff07-6de6-44f8-aa72-99ad6ef85e93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8903b338e5b4fc9b123c4f3696f1f33'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:51.358953 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-231dff07-6de6-44f8-aa72-99ad6ef85e93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8903b338e5b4fc9b123c4f3696f1f33'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:51.371857 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-231dff07-6de6-44f8-aa72-99ad6ef85e93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8903b338e5b4fc9b123c4f3696f1f33'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:51.385371 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1453/2897] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 15:12:50 2022] DELETE /identity/v3/users/e8903b338e5b4fc9b123c4f3696f1f33 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:12:51.390300 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019ed4f0-6fcb-4c5e-ba0f-89ccf7c84a59 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:51.390590 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019ed4f0-6fcb-4c5e-ba0f-89ccf7c84a59 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:51.390876 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019ed4f0-6fcb-4c5e-ba0f-89ccf7c84a59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:51.407917 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-019ed4f0-6fcb-4c5e-ba0f-89ccf7c84a59 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:12:51.409222 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-019ed4f0-6fcb-4c5e-ba0f-89ccf7c84a59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:12:51.468705 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1445/2898] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:12:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:12:51.478438 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231dff07-6de6-44f8-aa72-99ad6ef85e93 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:51.479874 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-231dff07-6de6-44f8-aa72-99ad6ef85e93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:12:51.538729 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231dff07-6de6-44f8-aa72-99ad6ef85e93 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:51.542536 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb83dbe0-484e-4180-91a4-fc1980583833 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:51.543225 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb83dbe0-484e-4180-91a4-fc1980583833 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:51.543514 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb83dbe0-484e-4180-91a4-fc1980583833 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:51.543797 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb83dbe0-484e-4180-91a4-fc1980583833 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:51.554960 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb83dbe0-484e-4180-91a4-fc1980583833 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:51.555897 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb83dbe0-484e-4180-91a4-fc1980583833 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:51.566314 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1454/2899] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:12:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 15:12:52.014981 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019ed4f0-6fcb-4c5e-ba0f-89ccf7c84a59 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:52.016641 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-019ed4f0-6fcb-4c5e-ba0f-89ccf7c84a59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:12:52.030129 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019ed4f0-6fcb-4c5e-ba0f-89ccf7c84a59 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:52.034980 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46418bff-d223-4815-861b-8ed8527085a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:52.035848 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46418bff-d223-4815-861b-8ed8527085a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:52.036172 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46418bff-d223-4815-861b-8ed8527085a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:52.036452 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46418bff-d223-4815-861b-8ed8527085a0 admin admin] PATH_INFO: `/v3/projects/9b4f2c2c197441b5ab316ffe1037de39` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:52.037633 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46418bff-d223-4815-861b-8ed8527085a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b4f2c2c197441b5ab316ffe1037de39)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:52.039527 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46418bff-d223-4815-861b-8ed8527085a0 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:52.144989 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-46418bff-d223-4815-861b-8ed8527085a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b4f2c2c197441b5ab316ffe1037de39', 'request_id': 'req-46418bff-d223-4815-861b-8ed8527085a0'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:52.202013 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-46418bff-d223-4815-861b-8ed8527085a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b4f2c2c197441b5ab316ffe1037de39', 'request_id': 'req-46418bff-d223-4815-861b-8ed8527085a0'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:52.207806 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-46418bff-d223-4815-861b-8ed8527085a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b4f2c2c197441b5ab316ffe1037de39', 'request_id': 'req-46418bff-d223-4815-861b-8ed8527085a0'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:52.209665 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:12:52.219436 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-46418bff-d223-4815-861b-8ed8527085a0 admin admin] The token cache is being invalidate because project 9b4f2c2c197441b5ab316ffe1037de39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:12:52.219765 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-46418bff-d223-4815-861b-8ed8527085a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:52.220433 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-46418bff-d223-4815-861b-8ed8527085a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:12:52.222067 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1446/2900] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 15:12:52 2022] DELETE /identity/v3/projects/9b4f2c2c197441b5ab316ffe1037de39 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:12:52.227125 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e672cecb-4193-42e4-9897-359f07f729c0 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:52.227416 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e672cecb-4193-42e4-9897-359f07f729c0 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:52.227693 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e672cecb-4193-42e4-9897-359f07f729c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:52.245479 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-e672cecb-4193-42e4-9897-359f07f729c0 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:12:52.246747 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e672cecb-4193-42e4-9897-359f07f729c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:12:52.306304 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1455/2901] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:12:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:12:52.314041 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d886197a-2c8b-498c-a07d-e9dde377dd48 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:52.314327 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d886197a-2c8b-498c-a07d-e9dde377dd48 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:52.314602 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d886197a-2c8b-498c-a07d-e9dde377dd48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:52.334229 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-d886197a-2c8b-498c-a07d-e9dde377dd48 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:12:52.335440 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-d886197a-2c8b-498c-a07d-e9dde377dd48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:12:52.344204 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1447/2902] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:12:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:12:52.348403 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e672cecb-4193-42e4-9897-359f07f729c0 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:52.358242 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e672cecb-4193-42e4-9897-359f07f729c0 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:52.362922 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566295d0-41f4-486b-b25d-9c0c435fccee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:52.363577 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566295d0-41f4-486b-b25d-9c0c435fccee admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:52.363863 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566295d0-41f4-486b-b25d-9c0c435fccee admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:52.364175 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566295d0-41f4-486b-b25d-9c0c435fccee admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:52.364939 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566295d0-41f4-486b-b25d-9c0c435fccee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:12:52.365250 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566295d0-41f4-486b-b25d-9c0c435fccee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:52.366846 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566295d0-41f4-486b-b25d-9c0c435fccee admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:52.378944 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1456/2903] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jun 15 15:12:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:12:52.383278 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d886197a-2c8b-498c-a07d-e9dde377dd48 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:52.393123 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d886197a-2c8b-498c-a07d-e9dde377dd48 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:52.397840 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2488b850-2668-456d-b637-25e3c5163aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:52.398494 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2488b850-2668-456d-b637-25e3c5163aec admin admin] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:52.398803 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2488b850-2668-456d-b637-25e3c5163aec admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:52.399088 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2488b850-2668-456d-b637-25e3c5163aec admin admin] PATH_INFO: `/v3/projects` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:52.399777 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2488b850-2668-456d-b637-25e3c5163aec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:52.401759 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2488b850-2668-456d-b637-25e3c5163aec admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:52.488521 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1448/2904] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jun 15 15:12:52 2022] POST /identity/v3/projects => generated 377 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:12:52.492644 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566295d0-41f4-486b-b25d-9c0c435fccee admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:52.503743 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566295d0-41f4-486b-b25d-9c0c435fccee admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:52.551372 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:12:52.561844 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a311fa-1494-4b68-b828-b957bb8f70ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:52.562444 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a311fa-1494-4b68-b828-b957bb8f70ff admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:52.562729 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a311fa-1494-4b68-b828-b957bb8f70ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:52.563022 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a311fa-1494-4b68-b828-b957bb8f70ff admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:52.563717 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a311fa-1494-4b68-b828-b957bb8f70ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:52.565697 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a311fa-1494-4b68-b828-b957bb8f70ff admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:52.642763 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1457/2905] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jun 15 15:12:52 2022] POST /identity/v3/users => generated 289 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:12:52.646908 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2488b850-2668-456d-b637-25e3c5163aec admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:52.658038 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2488b850-2668-456d-b637-25e3c5163aec admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:52.662688 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3150ef0e-adb4-4c2a-9daa-b1b97be85d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:52.663346 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150ef0e-adb4-4c2a-9daa-b1b97be85d58 admin admin] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:52.663632 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150ef0e-adb4-4c2a-9daa-b1b97be85d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:52.663964 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150ef0e-adb4-4c2a-9daa-b1b97be85d58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:52.664722 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3150ef0e-adb4-4c2a-9daa-b1b97be85d58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:52.666319 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3150ef0e-adb4-4c2a-9daa-b1b97be85d58 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:52.676818 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1449/2906] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 15:12:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:12:52.680733 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a311fa-1494-4b68-b828-b957bb8f70ff admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:52.692058 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a311fa-1494-4b68-b828-b957bb8f70ff admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:52.697370 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb4eb56-dc12-4b99-9edf-74e5ff0bed81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:52.698029 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb4eb56-dc12-4b99-9edf-74e5ff0bed81 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:52.698344 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb4eb56-dc12-4b99-9edf-74e5ff0bed81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:52.698626 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb4eb56-dc12-4b99-9edf-74e5ff0bed81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:52.699349 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb4eb56-dc12-4b99-9edf-74e5ff0bed81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:52.701850 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb4eb56-dc12-4b99-9edf-74e5ff0bed81 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:52.712657 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1458/2907] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jun 15 15:12:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:12:52.717019 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3150ef0e-adb4-4c2a-9daa-b1b97be85d58 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:52.726797 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3150ef0e-adb4-4c2a-9daa-b1b97be85d58 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:52.731339 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c65bd7d-a604-44f5-baa4-66406ab33f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:52.732280 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c65bd7d-a604-44f5-baa4-66406ab33f96 admin admin] REQUEST_METHOD: `PUT` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:52.732566 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c65bd7d-a604-44f5-baa4-66406ab33f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:52.732847 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c65bd7d-a604-44f5-baa4-66406ab33f96 admin admin] PATH_INFO: `/v3/projects/3c8b82a8f8f14a33997658528a5921e9/users/df2790b908604703a32417943ecdc362/roles/9effaf496b874d01bd3f6d0555ef2d69` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:52.748977 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c65bd7d-a604-44f5-baa4-66406ab33f96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c8b82a8f8f14a33997658528a5921e9, user_id=df2790b908604703a32417943ecdc362, role_id=9effaf496b874d01bd3f6d0555ef2d69)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:52.751583 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c65bd7d-a604-44f5-baa4-66406ab33f96 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:52.822759 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1450/2908] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jun 15 15:12:52 2022] PUT /identity/v3/projects/3c8b82a8f8f14a33997658528a5921e9/users/df2790b908604703a32417943ecdc362/roles/9effaf496b874d01bd3f6d0555ef2d69 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:12:52.834744 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c797b0f0-b1c4-4f98-a80a-97698b255e50 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:52.835033 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c797b0f0-b1c4-4f98-a80a-97698b255e50 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:52.835310 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c797b0f0-b1c4-4f98-a80a-97698b255e50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:52.861804 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-c797b0f0-b1c4-4f98-a80a-97698b255e50 None None] MFA Rules not processed for user `df2790b908604703a32417943ecdc362`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:12:52.863016 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c797b0f0-b1c4-4f98-a80a-97698b255e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:12:52.928438 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1459/2909] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:12:52 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:12:52.940645 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c65bd7d-a604-44f5-baa4-66406ab33f96 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:12:52.942591 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-7c65bd7d-a604-44f5-baa4-66406ab33f96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:12:52.998797 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c65bd7d-a604-44f5-baa4-66406ab33f96 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:12:53.002331 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b91aa2-c7b2-4c6d-ad33-0897c2edd5e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e44f6cd435be4e03bc79cb3125d71451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:12:53.002976 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b91aa2-c7b2-4c6d-ad33-0897c2edd5e1 service cinder] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:12:53.003265 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b91aa2-c7b2-4c6d-ad33-0897c2edd5e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:12:53.003530 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b91aa2-c7b2-4c6d-ad33-0897c2edd5e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:12:53.011616 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b91aa2-c7b2-4c6d-ad33-0897c2edd5e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:12:53.013116 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b91aa2-c7b2-4c6d-ad33-0897c2edd5e1 service cinder] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:12:53.022012 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1451/2910] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:12:52 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:13:02.925497 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=43,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:13:03.017947 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=55,keystone:INSERT=2 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:13:21.064380 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c797b0f0-b1c4-4f98-a80a-97698b255e50 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:13:21.126476 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c797b0f0-b1c4-4f98-a80a-97698b255e50 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:13:21.130929 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8795a9-2b85-4c17-9563-73f74def00a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:13:21.131786 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8795a9-2b85-4c17-9563-73f74def00a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:13:21.132071 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8795a9-2b85-4c17-9563-73f74def00a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:13:21.132371 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8795a9-2b85-4c17-9563-73f74def00a2 admin admin] PATH_INFO: `/v3/users/df2790b908604703a32417943ecdc362` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:13:21.133533 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8795a9-2b85-4c17-9563-73f74def00a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df2790b908604703a32417943ecdc362)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:13:21.135516 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8795a9-2b85-4c17-9563-73f74def00a2 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:13:21.306280 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-dc8795a9-2b85-4c17-9563-73f74def00a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df2790b908604703a32417943ecdc362'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:13:21.306945 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-dc8795a9-2b85-4c17-9563-73f74def00a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df2790b908604703a32417943ecdc362'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:13:21.307552 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-dc8795a9-2b85-4c17-9563-73f74def00a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df2790b908604703a32417943ecdc362'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:13:21.373267 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-dc8795a9-2b85-4c17-9563-73f74def00a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df2790b908604703a32417943ecdc362'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:13:21.387093 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-dc8795a9-2b85-4c17-9563-73f74def00a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df2790b908604703a32417943ecdc362'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:13:21.404047 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1460/2911] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jun 15 15:13:21 2022] DELETE /identity/v3/users/df2790b908604703a32417943ecdc362 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:13:21.408650 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6dd47e-4e27-4790-8ffe-c91b799c9176 None None] REQUEST_METHOD: `POST` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:13:21.409143 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6dd47e-4e27-4790-8ffe-c91b799c9176 None None] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:13:21.409262 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6dd47e-4e27-4790-8ffe-c91b799c9176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:13:21.426112 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.auth.core [None req-0b6dd47e-4e27-4790-8ffe-c91b799c9176 None None] MFA Rules not processed for user `11d932a20a0f4d8b9ce5ee15655d936c`. Rule list: `[]` (Enabled: `True`). {{(pid=80601) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:13:21.427319 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-0b6dd47e-4e27-4790-8ffe-c91b799c9176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:13:21.487645 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1452/2912] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jun 15 15:13:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:13:21.499011 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8795a9-2b85-4c17-9563-73f74def00a2 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:13:21.500556 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-dc8795a9-2b85-4c17-9563-73f74def00a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:13:21.561030 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8795a9-2b85-4c17-9563-73f74def00a2 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:13:21.564644 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4f2b36-55f5-4397-a7dc-e3495e9f1799 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c739b61c8a8b453e979bda720bafebb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:13:21.565421 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4f2b36-55f5-4397-a7dc-e3495e9f1799 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:13:21.565706 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4f2b36-55f5-4397-a7dc-e3495e9f1799 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:13:21.565982 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4f2b36-55f5-4397-a7dc-e3495e9f1799 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:13:21.575759 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4f2b36-55f5-4397-a7dc-e3495e9f1799 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:13:21.577331 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4f2b36-55f5-4397-a7dc-e3495e9f1799 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:13:21.586854 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1461/2913] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:13:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:13:22.000044 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6dd47e-4e27-4790-8ffe-c91b799c9176 None None] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:13:22.001510 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-0b6dd47e-4e27-4790-8ffe-c91b799c9176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:13:22.013914 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6dd47e-4e27-4790-8ffe-c91b799c9176 None None] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:13:22.019498 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c81738d-54ec-483c-bb0e-b48fcfeb5e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11d932a20a0f4d8b9ce5ee15655d936c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '688b80ae31764832aed9d6e0f3d3660f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:13:22.021295 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c81738d-54ec-483c-bb0e-b48fcfeb5e01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:13:22.021584 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c81738d-54ec-483c-bb0e-b48fcfeb5e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:13:22.021863 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c81738d-54ec-483c-bb0e-b48fcfeb5e01 admin admin] PATH_INFO: `/v3/projects/3c8b82a8f8f14a33997658528a5921e9` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:13:22.023094 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c81738d-54ec-483c-bb0e-b48fcfeb5e01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c8b82a8f8f14a33997658528a5921e9)` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:13:22.025782 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c81738d-54ec-483c-bb0e-b48fcfeb5e01 admin admin] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:13:22.130788 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-9c81738d-54ec-483c-bb0e-b48fcfeb5e01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c8b82a8f8f14a33997658528a5921e9', 'request_id': 'req-9c81738d-54ec-483c-bb0e-b48fcfeb5e01'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:13:22.207167 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-9c81738d-54ec-483c-bb0e-b48fcfeb5e01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c8b82a8f8f14a33997658528a5921e9', 'request_id': 'req-9c81738d-54ec-483c-bb0e-b48fcfeb5e01'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:13:22.214139 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-9c81738d-54ec-483c-bb0e-b48fcfeb5e01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c8b82a8f8f14a33997658528a5921e9', 'request_id': 'req-9c81738d-54ec-483c-bb0e-b48fcfeb5e01'} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:13:22.225464 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-9c81738d-54ec-483c-bb0e-b48fcfeb5e01 admin admin] The token cache is being invalidate because project 3c8b82a8f8f14a33997658528a5921e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80601) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:13:22.225824 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-9c81738d-54ec-483c-bb0e-b48fcfeb5e01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:13:22.226450 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.notifications [None req-9c81738d-54ec-483c-bb0e-b48fcfeb5e01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80601) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:13:22.228249 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1453/2914] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jun 15 15:13:21 2022] DELETE /identity/v3/projects/3c8b82a8f8f14a33997658528a5921e9 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:13:24.690253 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4f2b36-55f5-4397-a7dc-e3495e9f1799 service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:13:24.693598 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-2a4f2b36-55f5-4397-a7dc-e3495e9f1799 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:13:24.718948 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:13:24.787355 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4f2b36-55f5-4397-a7dc-e3495e9f1799 service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:13:24.848799 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7983d24a-e303-4f68-96e8-ea234fbf778c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bb6671c19464424a6e9cd4519ac93e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:13:24.849443 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7983d24a-e303-4f68-96e8-ea234fbf778c service placement] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:13:24.849721 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7983d24a-e303-4f68-96e8-ea234fbf778c service placement] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:13:24.849996 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7983d24a-e303-4f68-96e8-ea234fbf778c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:13:24.851763 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-7983d24a-e303-4f68-96e8-ea234fbf778c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:13:24.864484 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7983d24a-e303-4f68-96e8-ea234fbf778c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:13:24.866055 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7983d24a-e303-4f68-96e8-ea234fbf778c service placement] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:13:24.877085 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1462/2915] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:13:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:13:32.222283 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:13:34.873392 dv-cinder-787296-10-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=57 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:18:28.698610 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c81738d-54ec-483c-bb0e-b48fcfeb5e01 admin admin] Authenticating user token {{(pid=80601) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:18:28.702101 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.fernet_utils [None req-9c81738d-54ec-483c-bb0e-b48fcfeb5e01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80601) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:18:28.715374 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c81738d-54ec-483c-bb0e-b48fcfeb5e01 admin admin] Validating token access rules against request {{(pid=80601) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:18:28.720403 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df1217c-eebc-4522-9551-0ca555c81060 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bb6671c19464424a6e9cd4519ac93e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3094ff8966f149bda784da6a51b1358f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80601) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:18:28.725018 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df1217c-eebc-4522-9551-0ca555c81060 service placement] REQUEST_METHOD: `GET` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:18:28.725018 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df1217c-eebc-4522-9551-0ca555c81060 service placement] SCRIPT_NAME: `/identity` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:18:28.725018 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df1217c-eebc-4522-9551-0ca555c81060 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80601) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:18:28.730258 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df1217c-eebc-4522-9551-0ca555c81060 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:18:28.731871 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df1217c-eebc-4522-9551-0ca555c81060 service placement] RBAC: Authorization granted {{(pid=80601) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:18:28.742101 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: [pid: 80601|app: 0|req: 1454/2916] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jun 15 15:18:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:18:38.737532 dv-cinder-787296-10-iscsi devstack@keystone.service[80601]: DEBUG dbcounter [-] [80601] Writing DB stats keystone:SELECT=6 {{(pid=80601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}